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Preface 


In  early  2013,  the  leadership  of  the  Federal  Voting  Assistance  Program  (FVAP)  com¬ 
missioned  the  RAND  National  Defense  Research  Institute  (NDRI)  to  undertake  a 
collaborative,  multiyear  work  program  known  formally  as  FVAP  and  the  Road  Ahead. 
The  project  was  established  to  assist  FVAP  in  aligning  its  strategy  and  operations  to 
better  serve  its  mission  and  stakeholders  and  to  strengthen  FVAP’s  capacity  to  set  its 
own  course,  greet  change,  and  communicate  its  role  in  the  voting  community. 

This  report,  which  documents  the  RAND  project  team’s  objective  analysis,  its 
work  with  FVAP,  and  the  results  of  both,  describes  an  iterative  process  of  engagement 
with  a  very  small  federal  agency  seeking  to  elevate  its  performance  and  prepared  to 
implement  change.  The  project  enabled  a  significant  realignment  of  the  agency’s  strat¬ 
egy  and  operations.  Thus,  it  demonstrates  the  potential  for  substantial,  timely  change 
through  a  highly  collaborative  working  relationship,  rooted  in  a  systematic  and  ana¬ 
lytically  grounded  research  design. 

The  RAND  project  team  worked  with  FVAP  to  compare,  reconcile,  and  align 
what  was  in  the  agency’s  strategy  and  typical  of  its  operations  and  what  should  be, 
through  an  evidence-based  approach  that  included  logic  modeling,  stakeholder  out¬ 
reach,  and  a  requirements  assessment.  The  team  delivered  recommendations  and  imple¬ 
menting  guidance  over  the  course  of  the  project,  and  FVAP  enacted  change  through¬ 
out.  As  a  consequence,  this  report  offers  final  recommendations  and  guidance  largely 
proposed  to  lock  in  and  build  on  gains. 

This  report  should  be  of  interest  to  agencies  seeking  to  improve  their  strategic 
and  operational  alignment  and  develop  a  better  understanding  of  the  methods  and 
processes  available  to  do  so,  to  those  involved  in  those  agencies’  oversight,  and  to  stake¬ 
holders  seeking  to  understand  changes  occurring  within  FVAP. 

In  addition  to  the  findings,  recommendations,  and  guidance  from  our  analy¬ 
sis,  this  report  describes  in  detail  the  collaborative  process  that  the  RAND  team  and 
FVAP  used  to  develop  the  analysis  and  implement  recommendations,  respectively.  We 
have  structured  the  document  to  meet  the  needs  of  members  of  FVAP’s  leadership  as 
they  convey  the  process  that  they  used  to  implement  major  organizational  change  and 
to  aid  the  agency  in  further  implementation  of  recommendations.  For  readers  with 
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greater  interest  in  the  underlying  method,  we  offer  a  shorter,  derivative  report  that 
focuses  on  the  iterative  application  of  the  logic  model. 

This  research  was  sponsored  by  FVAP  and  conducted  within  the  Forces  and 
Resources  Policy  Center  and  the  Acquisition  and  Technology  Policy  Center  of  NDRI, 
a  federally  funded  research  and  development  center  sponsored  by  the  Office  of  the  Sec¬ 
retary  of  Defense,  the  Joint  Staff,  the  Unified  Combatant  Commands,  the  Navy,  the 
Marine  Corps,  the  defense  agencies,  and  the  defense  Intelligence  Community. 

For  more  information  on  the  RAND  Forces  and  Resources  Policy  Center  or 
the  RAND  Acquisition  and  Technology  Policy  Center,  see  www.rand.org/nsrd/ndri/ 
centers/frp  or  www.rand.org/nsrd/ndri/centers/atp  or  contact  the  director  (contact 
information  is  provided  on  the  web  pages). 
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Summary 


Broadly  speaking,  the  Federal  Voting  Assistance  Program  (FVAP)  is  charged  with 
administering  the  Uniformed  and  Overseas  Citizens  Absentee  Voting  Act  (UOCAVA) 
of  1986  (Pub.  L.  99-410),  as  amended,  and  assisting  so-called  UOCAVA  voters  in 
voting  successfully.  These  voters  include  uniformed-service  members  and  their  families 
residing  outside  their  voting  jurisdictions  and  other  U.S.  citizens  residing  outside  the 
United  States.  FVAP  leadership  was  concerned  that  the  agency’s  mission  had  become 
blurred  over  time,  that  its  operations  might  have  fallen  out  of  step  with  its  mission,  and 
that  the  organization  would  benefit  from  a  more  strategic  approach  to  setting  goals, 
organizing  for  action,  and  allocating  resources. 

To  address  these  concerns,  FVAP  leadership  commissioned  the  RAND  National 
Defense  Research  Institute  (NDRI)  to  undertake  a  collaborative,  multiyear  work  pro¬ 
gram  known  formally  as  FVAP  and  the  Road  Ahead  or  informally  as  the  strategic 
focus  project.  FVAP  wanted  help  aligning  its  strategy  and  operations  to  better  serve 
its  mission  and  stakeholders  and  to  strengthen  its  capacity  to  set  its  own  course,  greet 
change,  and  communicate  its  role  in  the  voting  community. 

This  report  documents  our  (the  RAND  project  team’s)  objective  analysis,  our 
work  with  FVAP,  and  the  results  of  both.  It  describes  an  iterative  process  of  engage¬ 
ment  with  a  very  small  federal  agency  seeking  to  elevate  its  performance  and  prepared 
to  implement  change.  In  so  doing,  it  also  demonstrates  the  potential  for  substantial, 
timely  gains  through  a  highly  collaborative  working  relationship,  rooted  in  a  system¬ 
atic  and  analytically  grounded  research  design.  The  report  does  not  offer  readers  a 
cookbook  for  change,  but  it  suggests  a  generalizable,  replicable  approach. 

The  report  describes  the  project  in  terms  of  the  analytical  approach,  a  set  of  out¬ 
comes,  and  the  process  that  connected  them  throughout  the  engagement.  First,  it  pro¬ 
vides  an  overview  of  the  two  analytical  phases  of  the  project.  Second,  it  describes 
changes  unfolding  within  FVAP — some  in  direct  response  to  the  project  and  some 
emerging  contemporaneously  and  indicative  of  the  agency’s  desire  to  progress  rapidly. 
The  intent  of  the  report  is  to  convey  the  experience  of  the  project  as  much  as  the  out¬ 
comes  and,  thus,  to  provide  readers  with  insight  into  the  process  and  an  indication  of 
the  possible. 
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In  addition  to  the  findings,  recommendations,  and  guidance  from  our  analysis, 
this  report  describes  in  detail  the  collaborative  process  that  FVAP  and  we  used  to 
develop  the  analysis  and  implement  recommendations,  respectively.  We  have  struc¬ 
tured  the  document  to  meet  the  needs  of  members  of  FVAP’s  leadership  as  they  convey 
the  process  that  they  used  to  implement  major  organizational  change  and  to  aid  the 
agency  in  further  implementation  of  recommendations. 


Organizational  and  Policy  Contexts 

FVAP  is  a  federal  agency  of  fewer  than  20  permanent  full-time-equivalent  (FTE)  posi¬ 
tions  within  the  Defense  Human  Resources  Activity  (DHRA),  which  is  a  U.S.  Depart¬ 
ment  of  Defense  (DoD)  field  activity  of  the  Under  Secretary  of  Defense  for  Personnel 
and  Readiness.  FVAP  has  existed  in  some  form  since  1955  and  now  serves  voters  cov¬ 
ered  under  UOCAVA  and,  in  large  part,  administers  UOCAVA  provisions  and  devel¬ 
ops  related  policy. 

FVAP’s  charter  presents  complexities,  challenges,  and  opportunities,  some  stem¬ 
ming  from  the  size  and  breadth  of  its  customer  base  and  others  from  its  institutional 
position.  UOCAVA  voters  include  both  members  of  the  defense  community  (in  this 
case,  uniformed  absentee  voters  and  their  families)  and  overseas  citizens  (largely  with¬ 
out  connection  to  the  military).  These  disparate  and  geographically  dispersed  popula¬ 
tions  extend  over  seven  continents  and  55  states  and  territories  and  include  thousands 
of  voting  jurisdictions.  FVAP  works  with  a  range  of  military  and  civilian  organiza¬ 
tions  that  mirrors  the  diversity  and  dispersion  of  those  voters.  In  the  United  States  and 
around  the  world,  it  works  with  Voting  Assistance  Officers  (VAOs),  Installation  Voter 
Assistance  (IVA)  offices,  and  nongovernmental  organizations  (NGOs).  It  also  works 
with  state  and  local  election  officials  to  provide  a  bridge  between  these  entities  and 
UOCAVA  voters  when  needed.  FVAP’s  charter  and  its  DoD  placement  also  require 
that  it  straddle  the  domains  of  policy  and  operations,  which  itself  represents  both  a 
challenge  and  an  opportunity.  Although  FVAP  is  one  provider  of  voting  assistance  in 
a  much  larger  system,  it  has  unique  leadership  qualifications  deriving  from  its  charter, 
its  insight  to  policy  and  operations,  its  accumulated  knowledge  and  tools,  and  its  dedi¬ 
cated  resources. 

In  this  context,  FVAP  has  had  difficulty  stating  what  exactly  its  mission  is.  This 
difficulty  might  also  be  attributable,  in  part,  to  turnover  in  FVAP’s  leadership.  The 
agency  saw  four  directors  in  a  five-year  period,  from  2008  to  2013,  and,  at  the  launch 
of  this  project,  was  operating  under  an  acting  director. 


Summary  xiii 


Study  Purpose  and  Approach 

RAND  and  FVAP  began  deliberating  on  the  terms  of  the  engagement  well  in  advance 
of  the  start  of  the  project.  An  intense  period  of  preliminary  discussion  yielded  agree¬ 
ment  on  both  the  nature  of  the  working  relationship  and  the  method  of  analysis.  The 
participants  in  the  discussions  agreed  that  the  project  must  be  collaborative  and  itera¬ 
tive  to  ensure  its  relevance  and  timeliness  and  built  language  into  the  project  descrip¬ 
tion  (PD)  to  support  that  position  (RAND  Corporation,  2013);  specifically,  RAND 
and  FVAP  would  “work  closely,  cooperatively,  and  iteratively”  from  start  to  finish. 
The  participants  also  agreed  that  a  method  known  as  logic  modeling  would  serve  as 
the  project’s  analytical  foundation.  The  discussions  provided  the  participants  with  an 
opportunity  to  pattern  collaboration,  set  expectations  around  results  and  resources, 
and,  ultimately,  decide  whether  the  project  was  viable. 

Thus,  the  collaborative  process  took  shape  before  the  project  started.  Over  the  life 
of  the  project,  we  (the  RAND  project  team)  would  work  with  FVAP  in  frequent  face- 
to-face  meetings  to  share,  vet,  and  clarify  ideas  and  to  discuss  and  refine  the  details  of 
our  approach.  The  collaboration  would  enable  us  to  develop  a  full  and  mutual  under¬ 
standing  of  FVAP’s  needs,  to  better  gauge  and  adjust  our  tactics  to  meet  those  needs, 
and  to  rapidly  transfer  recommendations  to  FVAP  leadership  so  that  FVAP  could  apply 
them  and  implement  change. 

At  the  same  time,  the  project  required  objectivity.  The  language  of  the  PD  pro¬ 
vided  guidance  on  the  interplay  between  our  collaboration  with  FVAP  on  the  one 
hand  and  the  need  for  analytical  independence  on  the  other: 

In  the  course  of  undertaking  this  analysis,  RAND  NDRI  will . . .  [ojffer  “indepen¬ 
dent”  recommendations  and  guidance.  Cognizant  of  the  diversity  of  views — both 
internal  and  external — as  to  FVAP’s  role  and  responsibilities,  RAND  NDRI  will 
not  seek  to  arbitrate  among  them;  rather,  it  will  offer  its  own  independent  view  as 
to  how  FVAP  can  best  fulfill  its  mission  in  a  manner  that  is  consistent  with  legis¬ 
lative  mandates.  These  recommendations  will  be  “independent”  in  that  they  may 
differ  with  internally  and/or  externally  held  views;  nevertheless,  they  will  emerge 
from  a  process  of  extensive  internal  and  external  engagement. 

On  that  basis,  we  worked  with  FVAP  to  compare,  reconcile,  and  align  what  was 
in  the  agency’s  strategy  and  typical  of  its  operations  and  what  should  be.  FVAP  did  not 
commission  us  to  conduct  a  program  evaluation  but  to  catalyze  change. 

It  became  clear  early  on  that  we  would  need  to  help  the  agency  address  the  ambi¬ 
guity  of  its  mission  and,  in  particular,  to  achieve  a  common,  shared  understanding  of 
the  meaning  of  the  word  assistance. 

We  planned  the  project  for  four  phases,  including  two  analytical  phases.  We 
front-loaded  the  analysis  in  phases  1  and  2  to  enable  and  support  early  action  by  the 
agency. 
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In  phase  1,  we  conducted  a  preliminary  assessment  of  FVAP’s  role  in  the  voting 
community,  the  activities  it  undertakes,  the  means  by  which  it  undertakes  them,  and 
the  reasons  it  undertakes  them.  In  close  coordination  with  FVAP,  we 

•  developed  a  logic  model  to  benchmark  the  agency’s  strategy  and  operations 

•  engaged  with  some  of  FVAP’s  stakeholders  to  elicit  their  views  of  the  agency’s  role 
in  the  voting  community 

•  analyzed  the  agency’s  core  requirements,  as  manifested  in  U.S.  law  and  policy  on 
uniformed-  and  overseas- citizen  absentee-voting  assistance. 

The  benchmark  model  served  as  a  diagnostic  tool  that  led  us  down  the  paths  of 
outreach  and  analysis  that  followed,  by  surfacing  issues  that  merited  closer  consider¬ 
ation.  The  modeling  process  or  exercise  included  a  two-day  all-hands  workshop,  weekly 
office  hours,  and  several  small-group  sessions,  each  of  which  contributed  information 
necessary  to  understanding  FVAP  and  constructing  the  model.  The  exercise  yielded 
an  FVAP-centric  picture  of  the  agency.  We  augmented  it  with  stakeholder  perspectives 
and  a  closer  look  at  requirements  to  fill  holes  in  our  understanding  of  the  agency  and 
to  test  and  validate  the  model. 

This  assessment  produced  preliminary  recommendations  for  improving  the  agen¬ 
cy’s  strategic  focus  and  strengthening  its  operations  and  organizational  structure  and 
initial  guidance  for  implementing  change.  FVAP  was  able  to  act  on  the  findings  imme¬ 
diately  because  we  shared  them  as  they  emerged,  to  generate  debate  and  discussion. 

In  phase  2  of  the  project,  we  took  a  step  back  to  consider  FVAP’s  role  in  the  con¬ 
text  of  a  larger  voting  assistance  system;  examine  its  engagement  through  partnerships, 
research,  and  other  activities;  and  identify  opportunities  for  it  to  do  so  more  effectively. 
We  also  worked  with  FVAP  on  the  development  and  refinement  of  the  final,  robust 
logic  model.  The  model  served  as  a  blueprint  for  the  agency’s  reorganization,  which 
we  address  in  our  discussion  of  the  evidence  of  change.  Phase  2  of  the  project  likewise 
involved  close  collaboration  with  FVAP.  While  we  were  undertaking  our  analysis,  the 
agency  was  changing  in  response  to  that  analysis,  to  internally  recognized  needs,  and 
to  other  forces.  Our  approach  to  the  analysis  evolved  as  a  consequence. 

In  phase  3,  we  synthesized  the  results  of  the  engagement  and  delivered  our  final 
recommendations  and  guidance.  Given  the  extent  of  the  change  that  had  already 
occurred,  we  directed  our  attention  to  locking  in  and  building  on  gains. 

In  phase  4,  we  are  disseminating  findings. 

During  the  course  of  our  engagement,  another  RAND  project  team  was  work¬ 
ing  with  FVAP  to  develop  measures,  metrics,  and  a  dashboard  to  support  situational 
awareness  and  the  continuous  assessment  of  the  effectiveness  of  FVAP  activities. 
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Key  Findings  from  Phase  1 

The  logic-modeling  exercise,  stakeholder  outreach,  and  requirements  assessment  yielded 
findings  on  FVAP’s  mission,  operations,  and  organization  and  the  alignment  among 
them  from  each  of  the  three  different  perspectives. 

Mission  Ambiguity  and  Organizational  Challenges 

The  development  of  the  benchmark  logic  model  enabled  us  to  conceptualize  FVAP’s 
view  of  itself  as  consisting  of  distinct  streams  of  activities,  each  reaching  out  to  the 
UOCAVA  voter,  but  largely  dependent  on  the  actions  of  intermediaries,  including 
VAOs,  election  officials,  and  NGOs.  A  set  of  focal  points  emerged  from  the  modeling 
exercise,  relating  to  mission  ambiguity  and  other  organizational  challenges: 

•  Mission  ambiguity.  The  logic  model  drew  attention  to  the  lack  of  a  clearly 
articulated,  shared  mission  among  FVAP  personnel,  which,  in  turn,  appeared  to 
reflect  a  lack  of  consensus  within  FVAP  as  to  the  agency’s  purpose  and  its  role  in 
the  voting  community.  If  the  FVAP  staff  did  not  share  an  understanding  of  the 
agency’s  mission  or  priorities,  it  could  not  share  a  consistent  view  of  what  it  should 
be  doing  or  how  it  should  go  about  doing  it. 

•  Stovepiping  and  fragmentation.  The  picture,  both  literal  and  figurative,  embod¬ 
ied  in  the  logic  model  suggested  that  FVAP  was  functioning  as  loosely  connected, 
separately  managed  streams  of  activities — namely,  voter  assistance,  election-offi¬ 
cial  assistance,  and  institutional  support,  including  technology  services  and  com¬ 
munication  services. 

•  Inadequate  capacities  and  capabilities  in  some  areas.  Our  conversations  with 
FVAP  and  an  initial  review  of  strategic  and  other  documents  suggested  that 
FVAP  lacked  capacity  and  capabilities  in  some  organizationally  important  areas, 
including  those  relating  to  data  collection,  research,  and  analysis. 

•  Suboptimal  staffing  decisions  and  organizational  configuration.  The  final  set 
of  issues  pertained  to  FVAP’s  conceptualization  of  labor  and  its  internal  organiza¬ 
tion,  both  in  general  and  in  the  face  of  less  than  20  permanent  FTEs.  FVAP  did 
not  appear  to  recognize  that  the  decision  to  use  staff  in  one  line  of  operations  or 
activity  stream  was  also  a  decision  to  not  use  staff  in  another — thus,  establish¬ 
ing  or  acting  on  the  agency’s  priorities.  It  also  appeared  as  if  FVAP  might  be  top 
heavy,  with  a  large  share  of  staff  in  leadership  and  advisory  positions. 

Lack  of  Clarity  Among  Stakeholders  About  the  Federal  Voting  Assistance  Program's 
Role 

Next,  we  reached  out  to  FVAP’s  stakeholders,  including  congressional  staff,  election 
officials,  overseas-citizen  NGOs,  other  representatives  of  DoD  from  the  Office  of 
the  Secretary  of  Defense  and  the  services,  academics  and  technologists,  and  election 
advocates. 
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Our  analysis  of  the  stakeholders’  comments  yielded  two  overarching  observations. 
First,  the  comments  converged  around  a  handful  of  themes,  albeit  sometimes  reflect¬ 
ing  very  different  perspectives  on  the  underlying  concern  and  its  remedy.  Although 
the  stakeholders  recognized  that  FVAP  faces  institutional  constraints,  e.g.,  as  a  very 
small  agency  within  a  much  larger  national-security  agency,  they  suggested  that  FVAP 
should  do  the  following: 

•  Stick  to  a  single,  limited  set  of  activities  and  do  them  well. 

•  Recognize  UOCAVA  voters  as  customers. 

•  Set  a  high  bar  for  itself  and  others. 

•  Avoid  any  appearance  of  partisanship  or  advocacy. 

•  Devote  more  effort  to  communicating  with  stakeholders,  by  reaching  out  to  them 
more  regularly,  soliciting  and  listening  to  their  views  about  what  they  need  and 
how  FVAP  can  help  to  meet  these  needs,  and  being  more  responsive  to  their 
input. 

•  Conduct  its  business  more  transparently. 

•  Act  more  deliberately  with  its  and  others’  resources. 

•  Recognize  that  it  is  part  of  a  larger  system  and  that  its  actions  have  implications 
not  just  for  itself  but  also  for  others  in  the  system. 

•  Manage  institutional  constraints  more  effectively. 

Some  also  suggested  that  FVAP’s  engagement  with  stakeholders  had  been  driven 
too  much  by  the  agendas  of  individuals  within  the  agency  and  not  enough  by  the  mis¬ 
sion  of  the  agency. 

Although  nearly  everyone  suggested  that  FVAP  stick  to  a  single,  limited  set  of 
activities,  many  disagreed  on  the  content  of  that  set.  For  example,  some  stakeholders 
asserted  that  FVAP  should  be  working  to  ensure  that  voters  do  register,  while  others 
said  that  FVAP  should  be  working  to  ensure  that  they  can  register.  Nevertheless,  most 
stakeholders  endorsed  direct,  hands-on  assistance  as  appropriate  and  constructive. 

Second,  a  notable  number  of  stakeholders  lacked  interest  in  or  awareness  of  FVAP 
and  its  services  and  how  the  agency  or  its  services  might  relate  to  them. 

Room  for  Realignment  and  Latitude  to  Chart  a  Path 

Lastly,  we  sought  to  identify  the  minimum  set  of  activities  that  FVAP  must  undertake 
to  meet  specific,  direct  requirements,  to  which  we  refer  as  the  core,  and  to  improve 
our  understanding  of  how  FVAP’s  activities  related  to  that  core.  To  identify  the  core, 
we  turned  first  to  the  laws  governing  the  agency.  We  looked  for  both  gaps  in  coverage 
and  opportunities  to  redirect  resources.  An  activity  outside  the  core  might  contribute 
importantly  to  the  assistance  of  UOCAVA  voters,  either  directly  or  indirectly,  by  sup¬ 
porting  core  activities,  meeting  more-general  requirements,  or  filling  other  institu¬ 
tional  needs,  but  FVAP  might  have  latitude  to  consider  alternative  courses  of  action. 
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We  found  room  for  realignment  within  and  among  FVAP’s  activities  but  did  not 
identify  immediate  candidates  for  dissolution  or  chasms  in  coverage.  In  comparing 
FVAP  activities  with  the  law,  we  found,  for  example,  that  F  VAP  might  have  substantial 
flexibility  with  VAO  training,  the  website,  the  Voting  Assistance  Guide,  information 
dissemination  and  outreach,  and  data  collection  but  might  need  to  do  more  to  consult 
with  state  and  local  election  officials  and  ensure  their  awareness  of  the  requirements  of 
UOCAVA  and  to  report  to  Congress  on  effectiveness. 


Preliminary  Recommendations  and  Initial  Guidance 

At  the  close  of  phase  1,  we  summarized  our  findings  and  introduced  a  set  of  pre¬ 
liminary  recommendations  and  initial  guidance.  Our  analysis  strongly  suggested  that 
FVAP,  its  stakeholders,  and  the  law  were  each  seeing  the  world  differently.  We  sum¬ 
marize  their  perspectives  as  follows: 

•  FVAP  saw  itself,  in  large  part,  as  benefiting  UOCAVA  voters  through  intermedi¬ 
aries,  such  as  VAOs,  election  officials,  and  NGOs. 

•  Stakeholders  did  not  see  a  clear  path  from  FVAP  to  voting  success  (i.e.,  casting  a 
counted  ballot)  and  were  uncertain  about  what  FVAP  was  doing  or  why. 

•  The  law  saw  FVAP  doing  more  than  it  “must”  in  some  areas  and  possibly  less  than 
it  “must”  in  others. 

We  concluded  that  the  differences  among  the  three  perspectives  signaled  substan¬ 
tive  disconnects.  If,  for  example,  FVAP  saw  itself  as  operating  through  intermediaries, 
but  those  same  intermediaries  did  not  see  the  path  from  FVAP  to  them  or  to  UOCAVA 
voters,  it  seemed  likely  that  something  was  amiss. 

We  recast  the  key  findings  from  phase  1  as  actionable  recommendations  and 
guidance  for  FVAP  leadership.  Although  the  agency  began  acting  on  our  suggestions 
before  we  formally  delivered  them,  we  recount  them  here  to  provide  a  record  of  the 
project  and  a  means  of  gauging  the  changes  that  we  report  in  the  next  section.  A  clearer 
understanding  of  where  FVAP  was  at  the  start  of  the  project  should  help  shed  light  on 
where  it  is  now. 

Become  "One  FVAP" 

To  become  one  FVAP — a  coherent  whole  with  the  capability  to  function  as  such — the 
agency  would  need  to  do  three  things: 

•  Come  to  terms  with  its  mission.  This  would  go  beyond  crafting  a  mission  state¬ 
ment,  to  developing  a  common,  shared  understanding  of  the  agency’s  purpose 
and  priorities.  It  would  entail  establishing  a  set  of  primary  functions,  consist¬ 
ing  of  core,  supporting,  and  institutional  activities,  which,  in  turn,  should  derive 
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from  an  evidence-based  analysis  of  requirements,  needs,  and  potential  or  actual 
effectiveness. 

•  Integrate  and  shore  up  operations  to  support  that  mission.  Operational 
integration  could  occur  through  consolidating  resources  into  fewer  divisions  or 
through  taking  actions  to  strengthen  the  connective  tissues  of  the  organization, 
e.g.,  by  improving  internal  communications  or  promoting  cross-training  within 
and  across  activity  streams.  Cross-training  could  serve  two  purposes,  by  increas¬ 
ing  staff  members’  understanding  of  the  interrelatedness  of  FVAP  activities  and 
addressing  staffing  constraints.  To  address  remaining  capacity  and  capability  def¬ 
icits,  FVAP  would  need  to  assess  its  functional  needs  (e.g.,  skills  and  tools),  iden¬ 
tify  gaps,  and  take  actions  to  fill  them.  We  recommended  that  FVAP  consider 
professional  development  as  a  means  to  fill  some  of  the  gaps.  Professional  develop¬ 
ment  appeared  to  be  available  to  FVAP  staff  on  an  ad  hoc  basis,  but  FVAP  could 
address  it  more  comprehensively  and  systematically. 

•  Communicate  more  effectively  about  the  mission  and  organizational  pri¬ 
orities.  Once  FVAP  defined  its  mission  and  organized  around  it,  it  would  need 
to  communicate  its  direction  more  clearly  and  consistently,  both  internally  and 
externally.  Beyond  a  common,  shared  understanding  of  the  agency’s  mission  and 
priorities,  FVAP  staff,  including  leadership,  would  need  a  common,  shared  vocab¬ 
ulary  for  describing  them. 

Build  Trust  and  Strengthen  Relationships 

FVAP  would  also  need  to  strengthen  relationships  with  stakeholders.  Stronger  relation¬ 
ships  could  enable  FVAP  to  use  its  own  resources  more  effectively,  e.g.,  by  spending  less 
time  fighting  fires,  better  leveraging  the  resources  of  its  partners,  and,  ultimately,  better 
serving  its  customers,  especially  UOCAVA  voters.  Accomplishing  this  might  involve  a 
fundamental  transformation  in  FVAP’s  relationships,  one  requiring  a  re-envisioning  of 
the  agency’s  approach  to  its  partners  and  customers  as  active  participants,  not  as  pas¬ 
sive  operatives  or  actors.  More  concretely,  FVAP  would  need  to  communicate  with  its 
stakeholders  more  regularly,  listen  more  carefully,  and  respond  more  actively  to  their 
input.  We  suggested  that  FVAP  would  need  to  do  more  than  just  broadcast;  it  would 
need  to  receive  and  respond.  Although  it  faces  constraints  because  it  operates  in  a 
national  security  institution  (DoD),  FVAP  should  also  operate  as  openly  as  possible  by, 
for  example,  posting  data  when  they  become  available,  making  public  its  criteria  for 
decisionmaking,  and  alerting  stakeholders  to  plans  for  future  changes  so  that  they  can 
prepare  for  the  changes  and,  if  possible,  participate  in  the  changes. 

Embrace  a  Culture  and  Principles  of  Effectiveness 

Lastly,  we  recommended  that  FVAP  adopt,  internalize,  and  apply  a  culture  and  prin¬ 
ciples  of  effectiveness  throughout  its  operations,  beginning  with  its  work  on  mission, 
priorities,  and  activities.  We  suggested  that  FVAP  interpret  the  congressional  mandate 
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to  report  on  effectiveness  as  a  way  of  doing  business  more  generally.  For  example,  we 
suggested  routine  use  of  benefit-cost  assessment,  be  it  quantitative  or  qualitative,  in 
developing  and  implementing  projects,  assessing  the  merits  of  proposed  activities,  and 
thinking  about  daily  tasks.  Taking  on  this  mind-set  could  lead  to  a  more  rigorous  and 
thorough  consideration  of  the  implications  of  policy  and  actions  for  FVAP  and  the 
voting  community — particularly  if  FVAP  were  to  take  a  broad,  systemic  view  of  voting 
assistance  and  consider  the  benefits  and  costs  of  activities  throughout  the  community. 
However,  the  approach  might  also  require  a  better  understanding  of  the  voting  assis¬ 
tance  system  and  additional  analytical  means. 


Phase  2:  The  Federal  Voting  Assistance  Program's  Engagement  with 
Its  Operating  Environment 

In  this  phase  of  the  project,  we  took  a  step  back  and  considered  voting  assistance 
from  the  perspective  of  a  larger  voting  assistance  system,  to  identify  opportunities  for 
FVAP  to  improve  its  engagement  with  its  operating  environment,  defined  as  including 
its  customers,  partners,  and  other  stakeholders;  related  organizations;  and  supporting 
infrastructure  and  technology.  We  paid  further  attention  to  FVAP’s  research-related 
activities,  which  resulted  in  the  development  of  project-management  tools,  and  looked 
more  closely  at  FVAP’s  work  with  people  and  entities  serving  UOCAVA  voters,  par¬ 
ticularly  in  the  military. 

To  do  this,  we  mapped  the  system  writ  large  and  then  examined 

•  how  FVAP,  VAOs,  and  IVA  offices  serve  their  customers,  including  UOCAVA 
voters,  as  part  of  a  larger  system  of  voting  assistance 

•  the  informational  and  instructional  needs  of  the  VAOs  and  IVA  offices  that  serve 
UOCAVA  voters  in  different  institutional  settings  in  that  system. 

To  scope  the  system,  we  met  with  VAOs  and  IVA  office  staff,  conducted  site  visits, 
and  observed  VAO  training  sessions  at  nine  installations  in  the  Washington,  D.C., 
area;  Colorado;  and  California.  We  also  observed  a  remote  session  conducted  with  a 
consular  office  in  Athens,  Greece.  We  approached  site  selection  strategically  insomuch 
as  we  attempted  to  cover  each  of  the  services  and  a  variety  of  operational  and  adminis¬ 
trative  settings,  with  offices  led  by  active-duty  military  personnel  and  civilians,  mind¬ 
ful  of  possible  cultural  differences  across  services  and  between  military  and  civilian 
settings. 

In  the  course  of  phase  2,  we  also  neared  completion  of  the  final,  robust  logic 
model.  That  model  depicts  a  more-unified  FVAP,  consisting  of  a  consolidated  voter 
assistance  activity  stream  and  a  set  of  mission  support-like  activities,  informed  by  a 
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deeper  understanding  of  the  voting  assistance  system  and  a  consistent  interpretation  of 
the  agency’s  mission. 


Key  Findings  from  Phase  2 

In  exploring  the  system  and  FVAP’s  role  in  it,  we  found  that  it  consists  of  many  poten¬ 
tial  providers  of  voting  assistance  and  points  of  service  (physical  or  virtual)  to  which 
UOCAVA  voters  can  turn  for  help  in  the  voting  process  (Figure  S.l).  For  example,  an 
absentee  active-duty  military  voter  might  approach  a  YAO  with  questions  about  his 
or  her  voting  jurisdiction;  alternatively,  the  voter  might  reach  out  directly  to  a  state  or 
local  election  official  or  go  to  an  NGO’s  website.  In  this  context,  FVAP  is  one  provider 
among  many. 

In  addition,  we  found  the  following: 

•  Law,  policy,  and  the  market  shape  the  system. 

•  Connectivity  and  coordination  vary  across  the  system. 

•  The  system  and  its  parts  need  knowledge,  capability,  and  capacity  to  function. 

•  FVAP  faces  substantial  functional  limitations  in  the  system.  FVAP  has  a  sub¬ 
stantial  role  in  policymaking,  but  it  operates  largely  through  intermediaries  over 
which  it  exerts  little  direct  control — it  is  not  responsible  for  compliance,  and  it 
lacks  enforcement  authority.  Moreover,  it  has  surprisingly  few  opportunities  to 
reach  proactively  into  the  voting  assistance  system  on  a  regular  basis. 

•  FVAP  occupies  a  unique  position  in  the  system,  given  its  place  at  the  nexus  of 
policy  and  operations  and  its  knowledge,  tools,  and  dedicated  resources. 

•  FVAP  can  use  training  to  leverage  its  position  in  the  system.  Training  stands  out 
as  a  promising  means  to  engage  intermediaries,  particularly  VAOs  and  IVA  office 
staff,  more  directly  and  get  closer  to  voters.  Training  can  hold  value  not  just  as  an 
educational  tool  but  also  as  a  potential  source  of  ancillary  benefits. 

•  These  ancillary  benefits  could  include  promoting  the  system’s  functionality  by 
helping  to  build  networks,  raising  FVAP’s  profile  as  a  resource,  signaling  the 
importance  of  voting  assistance,  enabling  collection  of  information  about  operat¬ 
ing  conditions,  and  encouraging  connectivity  and  coordination  across  the  system. 

Fiowever,  whether  training  can  produce  ancillary  benefits  depends  partly  on  the 
training  modality — that  is,  whether  it  is  in  person  or  online — and  partly  on  steps  that 
FVAP  takes  to  actively  seek  them.  To  understand  the  state  of  play,  we  assessed  FVAP’s 
options  in  terms  of  benefits,  costs,  and  risks  of  alternative  modalities.  We  found  the 
following: 

•  The  benefits  of  in-person  training,  much  more  so  than  of  online  training,  can 
extend  beyond  those  of  the  initial  learning  experience  and  spread  across  the 


Figure  S.1 

Voting  Assistance  Opportunities  and  Paths  to  Success  for  Uniformed  and  Overseas  Citizens  Absentee  Voting  Act  Voters 


SOURCE:  RAND  staff  analysis  with  input  from  FVAP  staff. 

NOTE:  ADM  =  active-duty  military.  ETS  =  Electronic  Transmission  Service.  UVAO  =  unit  VAO.  IVAO  =  installation  VAO.  FWAB  =  Federal  Write-In  Absentee 
Ballot.  SSVR  =  Senior  Service  Voting  Representative.  SVAO  =  Service  Voting  Action  Officer.  CVAO  =  chief  VAO.  DOS  =  U.S.  Department  of  State.  PSA  = 
public  service  announcement. 


RAND  RR882-S.  1 


Summary  xxi 


xxii  The  Federal  Voting  Assistance  Program  and  the  Road  Ahead 


system  through  networking,  signaling,  and  other  methods,  but  online  training 
can  be  accessed  globally,  24/7. 

•  The  costs  of  in-person  training  are  likely  to  be  greater  for  everyone  than  those  of 
online  training,  especially  if  information  technology  (IT)  is  already  in  place  and 
available  on  installations. 

•  Each  mode  presents  risks  to  the  individual  learning  experience  insomuch  as  it 
might  fail  to  meet  the  learning  needs  of  those  who  prefer  the  other  mode,  present 
challenges  of  availability,  or  entail  environmental  distractions,  but  some  institu¬ 
tional  and  systemic  risks  pertaining  to  coordination  and  turnout  are  specific  to 
in-person  training. 

The  analysis  also  supported  continuance  of  a  mixed  strategy,  including  both 
in-person  and  online  training,  and  implementation  of  programmatic  improvements, 
including  substantial  streamlining  of  in-person  training  and  the  adoption  of  best  prac¬ 
tices  in  adult  learning  and  training  evaluation.  This,  in  turn,  would  require  invest¬ 
ments  in  professional  development  for  FVAP. 


Evidence  of  Change  Within  the  Federal  Voting  Assistance  Program 

During  the  study  period — both  in  response  to  our  analysis  and  through  the  orga¬ 
nization’s  own  initiatives — FVAP  began  to  change.  In  the  final  step  of  our  analysis, 
we  explored  the  extent  of  this  change  as  manifest  in  tangible  evidence.  We  assessed 
changes  in  relation  to  our  earlier  recommendations  about  FVAP’s  mission  and  orga¬ 
nization,  stakeholder  relationships,  and  effectiveness.  As  shown  in  Table  S.l,  in  some 
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Evidence  of  Change  in  Relation  to  Preliminary  Recommendations  and  Guidance 


General 

Recommendation 

Areas  of  Emphasis 

Evidence  of  Change 

Become  one  FVAP 

Come  to  terms  with  the 
mission 

•  Reorientation  of  the  mission  and  purpose 

•  Redesign  of  the  FVAP  website  and  its  content 
and  outreach  and  training  materials 

•  Reorganization  of  the  agency 

•  Reconfiguration  of  the  call  center  as  an  in- 
house  voting  assistance  center 

•  New  forms  of  engagement  with  states, 
potentially  including  ongoing  work  with  CSG 

•  Reassessment  of  DoDI  1000.04 

Integrate  and  shore  up 
operations 

•  Reorganization  of  the  agency 

•  Reconfiguration  of  the  call  center  as  an  in- 

house  voting  assistance  center  and  related 
cross-training  of  FVAP  staff 
•  Enrollment  in  professional  development  pro¬ 
grams,  including  training  in  human  resources 
and  organizational  management 
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Table  S.1 — Continued 


General 

Recommendation 

Areas  of  Emphasis 

Evidence  of  Change 

Become  one  FVAP, 
continued 

Sharpen  and  clarify  the 
message 

•  Reorientation  of  the  mission  and  purpose 

•  Redesign  of  the  FVAP  website  and  its  content 
and  outreach  and  training  materials 

Build  trust  and 

strengthen 

relationships 

Work  with  partners  and 
serve  customers 

•  Redesign  of  the  FVAP  website  and  its  content 
and  outreach  and  training  materials 

•  Reconfiguration  of  the  call  center  as  an  in- 
house  voting  assistance  center 

•  New  forms  of  engagement  with  states, 
potentially  including  ongoing  work  with  CSG 

•  Outreach  to  Department  of  Defense  Educa¬ 
tion  Activity  schools 

•  National  Association  of  Secretaries  of  State 
resolution  on  voting  information 

•  Continuation  of  OMB  process  for  form  (FPCA 
and  FWAB)  revisions 

Communicate  better  and 
more  regularly 

•  Redesign  of  the  FVAP  website  and  its  content 
and  outreach  and  training  materials 

•  Customer  feedback  on  voting  assistance 
center  operations 

•  New  forms  of  engagement  with  states, 
potentially  including  ongoing  work  with  CSG 

•  Enrollment  in  professional  development  pro¬ 
grams,  including  training  in  customer  service 
skills  and  techniques 

Operate  as  openly  as 
possible 

•  New  forms  of  engagement  with  states, 
potentially  including  ongoing  work  with  CSG 

•  Continuation  of  OMB  process  for  form  (FPCA 
and  FWAB)  revisions 

•  Development  of  research  briefs 

Embrace  a  culture 
and  principles  of 
effectiveness 

Engage  routinely  in 
benefit-cost  assessment 
or  employ  other 
analytical  methods 

•  Use  of  findings  from  the  2012  postelection 
report  to  Congress  (FVAP,  2013b) 

•  Development  of  a  dashboard* * 3 

•  Enrollment  in  professional  development 

programs,  including  training  in  organiza¬ 
tional  and  project  management,  strategic 
planning  and  tactical  execution,  performance 
measurement,  and  problem-solving  and  data 
analysis 

•  Staff  interest  in  use  of  project-management 
tools 


SOURCE:  RAND  staff  analysis;  see  Chapter  Five  of  this  report  for  a  complete  discussion  of  the  general 
recommendations  and  areas  of  emphasis. 

NOTE:  OMB  =  Office  of  Management  and  Budget.  FPCA  =  Federal  Post  Card  Application.  CSG  =  Council 
of  State  Governments. 

3  The  availability  of  metrics — and  other  information — from  the  voting  assistance  center  suggests 
another  potential  avenue  of  change. 

cases,  we  found  evidence  of  outright  change,  such  as  the  agency’s  redesign  of  its  web¬ 
site.  In  other  instances,  we  found  evidence  of  progress,  as  in  the  example  of  profes¬ 
sional  development  that  could  facilitate  change.  And  in  yet  other  cases,  we  found  evi- 
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dence  of  unevenness,  suggesting  that,  although  the  agency  has  moved  forward  in  many 
respects,  work  remains  to  be  done.  Highlights  of  the  evidence  included  the  following: 

•  the  reorientation  of  the  agency’s  mission  and  purpose.  FVAP’s  newly  revised 
statement  of  purpose  reflects — and  announces — the  agency’s  intent  to  focus  on 
service  to  UOCAVA  voters.  Moreover,  it  does  so  succinctly  and  clearly. 

•  the  reorganization  of  the  agency.  FVAP  reorganized  its  operations  on  the 
basis  of  stakeholder  outreach,  statistical  evidence  of  efficacy,  the  analysis  that  the 
RAND  team  provided  to  the  agency  in  phase  1  of  this  project,  and  the  final, 
robust  logic  model.  FVAP  has  consolidated  assistance  to  voters  and  election  offi¬ 
cials  and  supporting  technology  and  research  in  a  single  activity  stream.  A  heavy 
reliance  on  evidence  was  a  major  change  in  FVAP’s  approach  to  setting  course 
and  taking  action. 

•  a  redesign  of  the  FVAP  website  and  of  outreach  and  training  materials. 

The  redesign  of  the  website  and  related  materials  supports  the  reorientation  of 
the  agency’s  mission  and  better  aligns  with  core  requirements.  For  example,  the 
agency  has  made  it  easier  for  stakeholders  to  identify  their  domains,  added  a  page 
devoted  to  outreach  materials,  and  made  it  possible  to  search  reports  and  other 
resources.  We  also  found  some  glitches,  suggesting  the  need  for  either  relabeling 
or  more-active  upkeep. 

•  the  reconfiguration  of  the  call  center  as  an  in-house  voting  assistance  center. 

FVAP  has  received  near-universal  positive  feedback  from  customers  on  the  recon¬ 
figured  call  center,  which  has  brought  staff  closer  to  the  agency’s  mission,  is  increas¬ 
ing  staff  awareness  of  ground  conditions,  and  was  enabled  by  cross-training. 

•  new  forms  of  engagement  with  states,  potentially  including  work  with  CSG. 
The  agency  is  exploring  new,  more-targeted  modes  of  engaging  directly  with  the 
states  and  localities,  including  in-depth  conversations  and  troubleshooting  ses¬ 
sions,  and  is  working  with  CSG. 

•  the  reassessment  of  DoD  Instruction  (DoDI)  1000.04.  FVAP  is  using  the 
reassessment  to  craft  revised  guidance  that  aligns  more  closely  with  core  legal 
requirements  and  more  clearly  sets  out  the  responsibilities  of  each  participant  in 
the  system. 

•  enrollment  in  professional  development  programs.  FVAP  is  adopting  a  rou- 
tinized  approach  to  working  with  staff  to  identify  and  fill  training  needs  and 
has  extended  training  to  a  large  share  of  the  agency.  That  training  has  included 
courses  that  address  institutional  leadership  and  project  management,  financial 
management,  problem-solving  and  data  analysis,  website  development,  and  cus¬ 
tomer  service. 

As  Table  S.l  suggests,  much  of  the  evidence  cuts  across  our  preliminary  recom¬ 
mendations,  suggesting  the  implementation  of  mutually  reinforcing  measures. 
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We  also  note  that  some  of  the  evidence  shows  FVAP  taking  steps  on  its  own  ini¬ 
tiative  to  improve  its  operations,  including  its  relations  with  stakeholders.  For  example, 
efforts  to  reconfigure  the  call  center  predated  our  involvement  with  FVAP.  A  compari¬ 
son  of  FVAP’s  2011  and  2013  efforts  to  revise  two  required  forms — the  FPCA  and  the 
FWAB — suggests  a  second  example,  not  addressed  above.  Stakeholders  depicted  the 
2011  effort  as  one  in  which  FVAP  did  not  solicit  or  encourage  stakeholder  feedback 
and,  as  a  result,  incurred  some  animosity.  The  2013  effort,  in  which  FVAP  chose  to 
invoke  a  well-established  OMB-led  process,  was  more  proactive,  involved  systematic 
engagement  far  exceeding  that  of  earlier  processes,  and  led  to  stakeholder  buy-in  and 
product  improvements.  This  instance,  like  that  of  the  call  center,  shows  FVAP  taking 
steps  on  its  own  to  identify  better  ways  of  working  with  stakeholders  and  conducting 
business  more  generally. 


Building  on  and  Locking  in  Progress:  Final  Recommendations 

Although  there  is  evidence  of  positive  change  at  FVAP,  we  see  the  potential  for  both 
additional  gains  and  fatigue.  For  this  reason,  our  final  recommendations  and  guidance 
speak  to  locking  in  and  building  on  recent  progress. 

FVAP  needs  to  adhere  to  its  mission,  notwithstanding  the  near  certainty  of  some 
leadership  turnover,  the  push  and  pull  of  divergent  stakeholder  interests,  and  the  ebbs 
and  flows  of  election  cycles,  but  we  are  not  advocating  rigidity.  FVAP  must  be  able 
to  respond  to  emerging  needs,  some  foreseeable  and  some  not;  it  is  through  those 
responses  that  it  will  demonstrate  whether  it  knows  what  it  is  and  where  it  is  going. 

What  follows  are  suggestions  for  consolidating  and  advancing  recent  progress. 
Some  of  these  suggestions  are  within  FVAP’s  power  to  implement,  and  others  must  be 
taken  up  elsewhere  in  DoD,  at  higher  organizational  levels,  or  outside  DoD.  Turning 
first  to  the  suggestions  largely  within  FVAP’s  control,  we  urge  FVAP  to  continue  to  do 
the  following: 

•  Internalize — and  outwardly  project — the  mission.  Operate  in  accordance 
with  the  newly  redefined  mission,  and  present  it  consistently  as  part  of  the  agen¬ 
cy’s  public  face  through  the  website,  written  materials,  presentations,  and  actions. 
Throughout  this  report,  we  have  presented  mission  as  a  center  from  which  change 
could  and  did  flow,  but  the  agency’s  ability  to  stay  on  track  will  also  depend  on 
that  mission  as  an  anchor.  An  FVAP  that  has  a  strong  sense  of  self  is  an  FVAP 
that  is  less  likely  to  be  buffeted  by  turnover,  competing  demands,  and  the  stresses 
and  strains  of  an  election  cycle. 

•  Invest  in  leadership  and  staff  with  professional  development.  Provide  lead¬ 
ership  and  staff  with  the  training  necessary  to  forward  the  mission,  and  give 
them  the  opportunity  to  absorb  it.  Simply  put,  if  leadership  or  staff  lacks  essential 
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knowledge  and  skills,  the  agency  cannot  carry  out  its  mission.  As  noted,  FYAP 
has  made  great  strides  in  this  arena,  but  it  will  need  to  continue  the  effort  and 
must  do  so  systematically. 

•  Promote  organizational  cohesion.  Embrace  the  integrative  process  that  began 
with  the  reorganization  of  divisions  and  activity  streams  by  improving  com¬ 
munication  across  the  agency,  resisting  the  temptation  to  create  new  stovepipes, 
accounting  for  the  interrelatedness  of  actions  among  activity  streams,  and  recog¬ 
nizing  and  validating  individuals’  accomplishments.  Staff  must  be  pulled  into  the 
process  as  coparticipants  and,  in  an  organization  with  fewer  than  20  permanent 
FTEs,  should  be  able — with  cross-training — to  reach  across  the  agency  to  both 
backfill  and  surge. 

•  Engage  with  stakeholders  as  stakeholders.  Work  diligently  to  remember  that 
stakeholders  have  interests  of  their  own  that  merit  close  consideration  and  are 
not  a  passive  audience  or  extension  of  the  agency’s  operations.  This  requires  that 
FVAP  more  fully  embrace  two-way  communication  and  reach  out  to  the  civilian 
public. 

•  Take  calls  for  effectiveness  to  heart.  This  requires  that  FVAP  consider  the  ben¬ 
efits,  costs,  and  risks  of  its  actions  in  relation  to  its  mission,  requirements,  and 
operations  on  a  day-to-day  basis  and  not  just  from  its  internal  perspective  but  also 
from  the  perspectives  of  its  customers,  partners,  and  other  stakeholders. 

•  Develop  stronger  discernment  skills.  Become  more  adept  at  distinguishing 
between  good  and  bad  ideas  for  new  projects  or  activities,  whether  internally  or 
externally  generated.  To  the  extent  that  FVAP  takes  on  projects  or  activities  that 
look  attractive  but  do  not  square  with  its  mission,  requirements,  and  organiza¬ 
tional  means,  it  allows  itself  to  become  less  focused  and  more  susceptible  to  set¬ 
backs  and  surprises. 

•  Work  with  the  logic  model  as  a  living  document.  Keep  the  final,  robust  logic 
model  at  hand,  as  both  a  reminder  of  institutional  identity  and  a  tool  for  com¬ 
municating  with  staff  and  stakeholders  with  a  shared  vocabulary,  but  encourage 
FVAP  to  revisit  the  specification  and  details  of  the  model  over  time.  If  FVAP 
uncovers  a  better  way  of  advancing  its  purpose,  it  can  update  the  model  to  reflect 
the  change. 

•  Undertake  periodic  health  checks.  Institute  an  ongoing  series  of  prepro¬ 
grammed  health  checks,  preferably  with  external  input,  to  better  ensure  contin¬ 
ued  progress  and  to  circumvent  the  potential  for  fatigue  and  backsliding. 

For  more-specific  suggestions,  framed  as  opportunities  and  largely  deriving  from 
the  phase  2  analysis,  we  direct  the  reader  to  the  final  chapter  of  this  report. 

Throughout  this  discussion,  we  have  assumed  the  interest  and  participation  of 
FVAP’s  stakeholders,  both  within  DoD  and  beyond  its  confines,  but,  as  we  learned 
through  our  early  outreach  to  stakeholders,  they  are  not  always  present.  Although 
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FVAP  cannot  force  constructive  engagement,  it  can  take  steps  to  promote  it  by  creat¬ 
ing  an  environment  in  which  stakeholders  deem  their  participation  worthwhile.  FVAP 
must  convince  its  stakeholders  that  it  is  taking  their  views  into  account  and  that  par¬ 
ticipation  is  in  their  interest. 

Our  final  thoughts  on  locking  in  and  building  on  gains  concern  the  appointment 
of  FVAP’s  leadership — specifically,  its  director — and  speak  to  the  issue  of  organiza¬ 
tional  resilience.  The  director  of  FVAP  should  come  to  the  table  with  strong  adminis¬ 
trative  skills,  as  a  trained  leader.  If  not  already  well-versed  in  voting  assistance,  he  or 
she  can  learn  from  staff  with  deep  subject-matter  expertise.  FVAP  is  an  agency  that 
will  continue  to  benefit  from  agnostic  civilian  leadership,  e.g.,  in  the  form  of  a  pro¬ 
fessional  administrator.  Voting  is  an  inherently  political  construct,  subject  to  sharply 
divergent  views  on  the  means  of  conduct,  be  they  printed  or  electronic,  and  a  plethora 
of  other  hot-button  issues.  To  minimize  the  potentially  destabilizing  effects  of  leader¬ 
ship  change  and  to  reduce  stakeholders’  concerns  that  FVAP’s  engagement  is  driven 
too  much  by  the  agendas  of  individuals  and  not  enough  by  the  mission  of  the  agency, 
we  suggest  prioritizing  managerial  expertise  in  the  hiring  decision. 


Concluding  Remarks 

The  project  was  intended  to  help  FVAP  in  aligning  its  strategy  and  operations  to  better 
serve  its  mission  and  stakeholders  and  to  strengthen  FVAP’s  capacity  to  set  its  own 
course,  greet  change,  and  communicate  its  role  in  the  voting  community. 

Looking  back  over  the  life  of  this  project,  we  believe  that  FVAP  has  made  con¬ 
siderable  progress.  FVAP  has  reoriented  its  focus  and  reorganized  its  operations;  has 
redesigned  its  website  and  outreach  materials  to  enable  and  communicate  change;  has 
substantially  altered  its  approach  to  stakeholders;  and  is  investing  in  its  staff  to  build 
capabilities  and  capacity  to  support  the  reorientation  and  reorganization. 

We  cannot,  as  yet,  connect  FVAP’s  reorientation  or  reorganization  to  improve¬ 
ments  in  outcomes  for  UOCAVA  voters — the  changes  in  FVAP  are  too  recent  and 
largely  untested — but  the  steps  that  FVAP  has  taken  to  realign  its  mission  and  opera¬ 
tions  should  cut  a  clearer  path  from  the  program’s  activities  to  outcomes  and,  thus, 
better  support  those  outcomes. 

We  also  see  the  results  as  demonstrating  to  FVAP,  to  other  DoD  agencies,  and  to 
other  governmental  and  nongovernmental  agencies  that  they  can  take  concrete  action 
to  overcome  obstacles  to  change  and  place  themselves  among  the  minority  of  organiza¬ 
tions  that  implement  change  successfully.  As  both  experience  and  the  large  literature 
on  change  management  suggest,  organizational  change  is  difficult  but  still  attainable. 
The  collaboration  between  RAND  and  FVAP  began  before  the  start  of  the  project 
with  an  intense  period  of  discussion  around  the  terms  of  the  engagement.  The  project 
launched  with  an  inclusive  and  expansive  attempt  to  better  understand  the  organiza- 
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tion  from  its  perspective,  from  the  perspective  of  its  stakeholders,  and  from  the  per¬ 
spective  of  the  laws  and  policies  governing  it.  We  employed  a  standard,  time-tested, 
and  readily  available  method — namely,  logic  modeling — which  we  supplemented  with 
stakeholder  outreach,  a  requirements  assessment,  and  consideration  of  the  larger  voting 
assistance  system.  And,  the  collaborative  relationship  made  it  possible  for  FVAP  to 
make  changes  expeditiously.  We  can  claim  robustness,  relevance,  and  timeliness  but 
not  uniqueness.  Two  human  factors  also  stood  out  as  essential  to  progress:  the  agency’s 
commitment  to  implementing  change  from  the  outset  and  the  mutual  trust  that  an 
ongoing,  collaborative  relationship  can  engender. 
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active-duty  military 

BAA 

Broad  Agency  Announcement 

BCA 
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ETS 

Electronic  Transmission  Service 
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Federal  Post  Card  Application 

FRA 

Federal  Railroad  Administration 

FTE 

full-time  equivalent 
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FWAB 

GAO 

GSA 

HR 

IG 

IT 

IVA  office 

IVAO 

LA 
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Lx 

MOVE  Act 

MPSA 

n/a 

NASED 

NASS 

NDRI 

NGO 
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OMB 

OPM 

OSD 

PA 

PAM 

PD 


Federal  Voting  Assistance  Program 

Federal  Write-In  Absentee  Ballot 

U.S.  Government  Accountability  Office 

General  Services  Administration 

human  resources 

inspector  general 

information  technology 

Installation  Voter  Assistance  office 

installation  Voting  Assistance  Officer 

legislative  affairs 

learning  management  system 

level,  where  x  represents  1,  2,  3,  or  4  in  the  Kirkpatrick  model 
Military  and  Overseas  Voter  Empowerment 
Military  Postal  Service  Agency 
not  applicable 

National  Association  of  State  Election  Directors 

National  Association  of  Secretaries  of  States 

RAND  National  Defense  Research  Institute 

nongovernmental  organization 

National  Institute  of  Standards  and  Technology 

National  Voter  Registration  Act  of  1993 

Office  of  Management  and  Budget 

Office  of  Personnel  Management 

Office  of  the  Secretary  of  Defense 

public  affairs 

pamphlet 

project  description 
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ProfDev 

professional  development 

PSA 

public  service  announcement 

SME 

subject-matter  expert 

SSVR 

Senior  Service  Voting  Representative 

SVAO 

Service  Voting  Action  Officer 

TRADOC 

U.S.  Army  Training  and  Doctrine  Command 

UOCAVA 

Uniformed  and  Overseas  Citizens  Absentee  Voting  Act 

URL 

uniform  resource  locator 

U.S.C. 

U.S.  Code 

USD(P&R) 

Under  Secretary  of  Defense  for  Personnel  and  Readiness 

USPS 

U.S.  Postal  Service 

UVAO 

unit  Voting  Assistance  Officer 

VAG 

Voting  Assistance  Guide 

VAO 

Voting  Assistance  Officer 

VARK 

visual,  aural,  read  and  write,  and  kinesthetic 

WHS 

Washington  Headquarters  Services 

CHAPTER  ONE 


Introduction 


Broadly  speaking,  the  Federal  Voting  Assistance  Program  (FVAP)  is  charged  with 
administering  the  Uniformed  and  Overseas  Citizens  Absentee  Voting  Act  (UOCAVA) 
of  1986  (Pub.  L.  99-410),  as  amended,  and  assisting  so-called  UOCAVA  voters  in 
voting  successfully.  These  voters  include  uniformed-service  members  and  their  families 
residing  outside  their  voting  jurisdictions1  and  other  U.S.  citizens  residing  outside  the 
United  States.  FVAP  leadership  was  concerned  that  the  agency’s  mission  had  become 
blurred  over  time,  that  its  operations  might  have  fallen  out  of  step  with  its  mission,  and 
that  the  organization  would  benefit  from  a  more  strategic  approach  to  setting  goals, 
organizing  for  action,  and  allocating  resources. 

To  address  these  concerns,  FVAP  leadership  commissioned  the  RAND  National 
Defense  Research  Institute  (NDRI)  to  undertake  a  collaborative,  multiyear  work  pro¬ 
gram  known  formally  as  FVAP  and  the  Road  Ahead  and  informally  as  the  strategic 
focus  project.  FVAP  wanted  help  aligning  its  strategy  and  operations  to  better  serve 
its  mission  and  stakeholders  and  to  strengthen  its  capacity  to  set  its  own  course,  greet 
change,  and  communicate  its  role  in  the  voting  community. 

This  report  documents  our  (the  RAND  project  team’s)  objective  analysis,  our 
work  with  FVAP,  and  the  results  of  both.  It  describes  an  iterative  process  of  engage¬ 
ment  with  a  very  small  federal  agency  seeking  to  elevate  its  performance  and  prepared 
to  implement  change.  In  so  doing,  the  report  also  demonstrates  the  potential  for  sub¬ 
stantial,  timely  gains  through  a  highly  collaborative  working  relationship,  rooted  in  a 
systematic  and  analytically  grounded  research  design.  The  report  does  not  offer  readers 
a  cookbook  for  change,  but  it  suggests  a  generalizable,  replicable  approach. 


1  As  defined  in  UOCAVA,  the  phrase  “absent  uniformed  services  voters”  includes  spouses  and  dependents  who, 
by  reason  of  the  active  duty  or  service  of  the  member,  are  absent  from  the  place  of  residence  where  the  spouse  or 
dependent  is  otherwise  qualified  to  vote.  Such  a  voter  could  be  located  in  the  United  States  or  overseas. 
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The  sections  that  follow  describe  some  of  the  challenges,  complexities,  and  oppor¬ 
tunities2  that  we  encountered  and  discuss  the  terms  of  the  engagement,  including  the 
nature  of  the  collaboration. 


Challenges,  Complexities,  and  Opportunities 

FVAP  is  a  federal  agency3  of  fewer  than  20  permanent  full-time-equivalent  (FTE) 
staff  members,  housed  in  the  Defense  Human  Resources  Activity  (DHRA),  which  is 
a  U.S.  Department  of  Defense  (DoD)  field  activity  of  the  Under  Secretary  of  Defense 
for  Personnel  and  Readiness  (USD[P&R])  in  the  Office  of  the  Secretary  of  Defense 
(OSD).  FVAP  has  existed  in  some  form  since  1955  and  now  serves  voters  covered  under 
UOCAVA  and,  in  large  part,  administers  the  provisions  of  UOCAVA  and  develops 
related  policy. 

A  combination  of  executive  order  (EO),  DoD  directive  (DoDD),  and  DoD 
instruction  (DoDI)  has  delegated  administrative  responsibility  to  FVAP,  through 
a  channel  that  starts  with  the  Secretary  of  Defense  as  the  presidential  designee  for 
UOCAVA  voting  (see  EO  12642  [Reagan,  1988])  and  continues  through  USD(P&R) 
to  DHRA  before  reaching  FVAP.  Because  of  FVAP’s  placement  in  an  OSD  field  activ¬ 
ity,  the  agency  engages  in  policy  development  and  deals  with  operations.  For  this  and 
other  reasons,  it  faces  complex  logistical,  organizational,  and  institutional  challenges, 
which  we  describe  below. 

Disparate  and  Diffuse  Populations 

FVAP  serves  disparate  and  geographically  diffuse  populations,  extending  over  seven 
continents,  55  states  and  territories,  and  thousands  of  voting  jurisdictions.  FVAP’s 
responsibilities  extend  not  just  to  the  defense  community  but  also  to  overseas  citizens 
without  any  connection  to  the  military.  Whereas  other  DoD  agencies,  such  as  those 
dealing  with  issues  of  base  realignment  and  closure,  infrastructure  planning  and  devel¬ 
opment,  or  education  and  health  affairs,  might  routinely  reach  out  to  civilians  with 
either  a  tight  or  loose  connection  to  the  military,  e.g.,  as  contractors  or  neighbors, 
FVAP  serves  overseas  citizens  simply  by  virtue  of  their  overseas  residence  and  voting 
eligibility. 


2  We  did  not  fully  appreciate  the  challenges,  complexities,  and  opportunities  facing  FVAP  until  we  had  com¬ 
pleted  much  of  our  analysis  of  FVAP’s  strategy,  operations,  and  operating  environment.  We  address  most  of  the 
underlying  issues  in  greater  depth  in  later  chapters,  but  we  introduce  them  here  to  orient  our  readers. 

3  Here,  we  use  the  word  agency  colloquially  to  denote  the  FVAP  office  and  its  staff,  as  distinct  from  the  term 
federal  voting  assistance  program ,  which  can  be  used  to  describe  all  federal  voting  assistance. 
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Reliance  on  Partners  and  Dynamism  of  System 

As  a  very  small  agency  tasked  with  administering  a  law  that  serves  millions  of  voters, 
FVAP  does  not  and  cannot  go  it  alone.  FVAP  works  with  a  range  of  military  and  civil¬ 
ian  organizations  that  mirror  the  disparate  and  diffuse  populations  that  constitute 
UOCAVA  voters.  In  the  United  States  and  around  the  world,  it  works  with  Voting 
Assistance  Officers  (VAOs),  Installation  Voter  Assistance  (IVA)  offices,  and  nongov¬ 
ernmental  organizations  (NGOs).  It  also  works  with  state  and  local  election  officials  to 
provide  a  bridge  between  these  entities  and  UOCAVA  voters  when  needed. 

•  VAOs  are  expected  to  ensure  that  citizens  covered  by  UOCAVA  understand  their 
voting  rights  and  how  to  register  and  vote  absentee  and  to  provide  accurate  non¬ 
partisan  voting  information  and  assistance.4  They  can  be  members  of  the  uni¬ 
formed  services,  civilians,  or  members  of  overseas  U.S.  citizen  organizations. 

•  The  Military  and  Overseas  Voter  Empowerment  (MOVE)  Act  of  2009  (Pub. 
L.  111-84)  specifies  that  IVA  offices  be  established  on  installations  to  assist  absent 
uniformed-service  members  and  their  families  with  voting-related  matters.5 
The  offices,  as  voter-registration  agencies  under  the  National  Voter  Registration 
Act  of  1993  (Pub.  L.  103-31)  (NVRA),  are  also  responsible  for  providing  voter- 
registration  assistance  to  uniformed-service  members  and  their  families,  overseas 
citizens,  and  all  other  civilian  voters  who  can  visit  the  offices.6 

Changes  in  the  composition  of  the  voting  assistance  system  and  in  the  relation¬ 
ships  among  the  players  add  to  the  challenges.  For  example,  the  MOVE  Act  added  the 
IVA  offices  to  the  system  and,  in  so  doing,  altered  the  architecture  of  the  system. 

Competing  Priorities 

Moreover,  as  many  of  FVAP’s  stakeholders  are  quick  to  note,  the  act  of  voting  is  not  a 
front-burner  issue  for  DoD,  at  least  not  in  the  years  between  general  elections.  FVAP 
supports  absent  uniformed-service  voters  and  their  families,  hence  its  placement  in 
USD(P&R),  but  the  role  is  not  central  to  DoD’s  mission,  which  is  “to  provide  the  mili¬ 
tary  forces  needed  to  deter  war  and  to  protect  the  security  of  our  country.”7 

Implications  for  the  Mission 

In  the  context  of  these  complexities  and  challenges,  FVAP  has  had  difficulty  stat¬ 
ing  exactly  what  its  mission  is.  This  difficulty  might  also  be  attributable,  in  part,  to 


4  Information  on  VAOs  is  available  at  FVAP,  undated  (d). 

5  For  a  more-complete  discussion  of  the  MOVE  Act  and  IVA  offices,  see  FVAP,  undated  (a),  pp.  5,  8. 

6  This  report  focuses  on  VAOs  who  are  affiliated  with  the  military  or  the  U.S.  Department  of  State  and  on  the 
services  provided  by  IVA  offices  to  uniformed-service  members  and  their  families. 

7  This  statement  was  posted  at  DoD,  undated,  as  of  July  24,  2014. 
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turnover  in  FVAP’s  leadership.  The  agency  saw  four  directors  in  a  five-year  period, 
from  2008  to  2013,  and  at  the  launch  of  this  project  was  operating  under  an  acting 
director — who  was  the  first  of  two  in  that  same  period.  The  second  acting  director 
became  the  permanent  director  several  months  into  the  project  and  has  remained  in 
that  position  throughout  our  engagement,  suggesting  the  potential  to  impart  longer- 
term  institutional  stability. 

Table  1.1  presents  some  of  the  different  mission  statements  that  FVAP  posted  or 
considered  during  the  2008-2013  period. 

Since  this  project  began,  FVAP  has  adopted  a  statement  of  purpose  that  is  con¬ 
sistent  with  a  focus  on  more-direct,  hands-on  assistance,  which  we  present  in  Chapter 
Seven;  however,  we  argue  that  the  issue  is  less  about  words  per  se  than  about  interpre¬ 
tation.  Any  one  of  the  earlier  statements  could  have  led  FVAP  to  the  same  place,  but 
none  did.8 

Looking  forward,  we  emphasize  that  FVAP  is  one  provider  of  UOCAVA  voting 
assistance  among  many  and  that  often  it  depends  on  others  to  reach  its  customers,  but 
it  also  has  unique  qualifications  that  separate  it  from  other  providers  in  the  federal  gov- 

Table  1.1 

Recent  Federal  Voting  Assistance  Program  Mission  Statements 


Source  of  Statement 


Excerpt  of  Statement 


Strategic  plan,  2005-2010 
(FVAP,  2005a) 


The  mission  of  the  Federal  Voting  Assistance  Program  is  to: 

•  Inform  and  educate  U.S.  citizens  worldwide  of  their  right  to  vote; 

•  Foster  voting  participation; 

•  Protect  the  integrity  of,  and  enhance,  the  electoral  process  at  the 
Federal,  state  and  local  levels;  and 

•  Promote  effectiveness  and  efficiency  in  the  administration  of  the 
Uniformed  and  Overseas  Citizens  Absentee  Voting  Act. 


Strategic  plan,  2010-2017 
(FVAP,  2011e);  also  posted  on 
the  FVAP  website  in  2013 


The  Federal  Voting  Assistance  Programs  exists  to: 

•  Assist  uniformed  services  and  overseas  voters  exercise  their  right 
to  vote  so  that  they  have  an  equal  opportunity  with  the  general 
population  to  have  their  vote  counted. 

•  Assist  the  States  in  complying  with  relevant  federal  laws,  and 
advise  them  on  ways  to  best  comply. 

•  Advocate  on  behalf  of  the  uniformed  services  and  overseas 
voters,  identifying  impediments  to  their  ability  to  exercise 
their  right  to  vote,  and  proposing  methods  to  overcome  those 
impediments. 


Draft  language  under  The  Federal  Voting  Assistance  Program  is  the  lead  Federal  agency 

consideration  in  the  early  providing  voter  assistance  to  uniformed  and  overseas  citizens.  FVAP 

spring  of  2013a  provides  information  and  tools  to  assist  voters  to  cast  their  ballot  and 

advises  and  assists  States  in  complying  with  relevant  Federal  laws. 


SOURCES:  FVAP,  2005a;  FVAP,  2011e;  and  an  unpublished  draft,  from  early  spring  of  2013. 
a  The  draft  mission  statement  was  under  consideration  in  the  early  spring  of  2013  but  had  not  been 
fully  discussed  within  the  agency  or  formally  adopted  at  that  time. 


As  evidence  of  this  point,  the  new  formulation  draws  on  some  of  the  same  vocabulary  as  the  old. 


Introduction  5 


ernment  and  elsewhere.  It  is  the  only  public  agency  chartered  through  law,  directives, 
and  instructions  with  the  primary  purpose  of  assisting  UOCAYA  voters,  and,  over 
time,  it  has  become  a  comprehensive  repository  of  information  on  the  processes,  tools, 
and  resources  for  UOCAVA  voting.  It  is  also  responsible  for  developing  and  maintain¬ 
ing  those  processes,  tools,  and  resources.  One  might  contend  that  it  is  the  only  public 
agency  with  the  credentials,  internal  motivation,  and  dedicated  resources  to  be  a  leader 
on  UOCAVA  voting  in  the  voting  assistance  system.  Moreover,  in  sitting  at  the  nexus 
of  policy  and  operations,  FVAP  has,  on  the  one  hand,  a  substantial  role  in  policymak¬ 
ing  and,  on  the  other  hand,  a  sweeping  view  of  the  system.  The  latter  suggests  potential 
for  a  high  degree  of  environmental  awareness. 


Terms  of  Engagement 

RAND  and  FVAP  began  deliberating  on  the  terms  of  this  engagement  well  in  advance 
of  the  start  of  the  project.  An  intense  period  of  preliminary  discussion,  which  lasted 
several  weeks,  yielded  agreement  on  both  the  nature  of  the  working  relationship  and 
the  method  of  analysis.  The  participants  in  that  discussion  agreed  that  the  project 
must  be  collaborative  and  iterative  to  ensure  its  relevance  and  timeliness  and  worked 
together  to  develop  a  project  description  (PD)  to  support  that  position;  specifically, 
RAND  and  FVAP  would  “work  closely,  cooperatively,  and  iteratively”  (RAND  Cor¬ 
poration,  2013)  from  start  to  finish.  The  participants  also  agreed  that  logic  modeling, 
which  is  the  focus  of  Chapter  Two  and  Appendix  A,9  would  provide  the  analytical 
foundation  for  the  engagement. 

Beyond  generating  the  PD,  the  discussions  provided  the  participants  with  an 
opportunity  to  pattern  collaboration  and  to  learn  more  about  each  other’s  expectations, 
weigh  associated  resource  requirements,  determine  whether  they  would  be  able  to  work 
together  constructively,  and,  ultimately,  decide  whether  the  project  was  viable. 

Thus,  the  collaborative  process  took  shape  before  the  project  started.  Over  the 
life  of  the  project,  we — the  RAND  project  team — would  work  with  FVAP  in  fre¬ 
quent  face-to-face  meetings  to  share,  vet,  and  clarify  ideas  and  to  discuss  and  refine  the 
details  of  our  approach.  The  collaboration  would  enable  us  to  develop  a  full,  mutual 
understanding  of  FVAP’s  needs,  better  gauge  and  adjust  our  tactics  to  meet  those 
needs,  and  rapidly  transfer  recommendations  to  FVAP  leadership  so  that  they  could 
apply  them  and  implement  change. 


9  As  addressed  in  Chapter  Two  and  Appendix  A,  logic  models,  which  often  look  like  flow  charts,  can  be  used 
to  articulate  the  critical  path  by  which  a  program  seeks  to  achieve  outcomes  and  attain  its  mission,  construct 
a  well-aligned  strategy  that  supports  its  mission,  communicate  internally  and  externally,  and  conduct  program 
evaluations. 
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At  the  same  time,  the  project  required  objectivity.  The  language  of  the  PD  pro¬ 
vided  guidance  on  the  interplay  between  our  collaboration  with  FVAP  on  the  one 
hand  and  the  need  for  analytical  independence  on  the  other: 

In  the  course  of  undertaking  this  analysis,  RAND  NDRI  will . . .  [ojffer  “indepen¬ 
dent”  recommendations  and  guidance.  Cognizant  of  the  diversity  of  views — both 
internal  and  external — as  to  FVAP’s  role  and  responsibilities,  RAND  NDRI  will 
not  seek  to  arbitrate  among  them;  rather,  it  will  offer  its  own  independent  view  as 
to  how  FVAP  can  best  fulfill  its  mission  in  a  manner  that  is  consistent  with  legis¬ 
lative  mandates.  These  recommendations  will  be  “independent”  in  that  they  may 
differ  with  internally  and/or  externally  held  views;  nevertheless,  they  will  emerge 
from  a  process  of  extensive  internal  and  external  engagement. 

On  that  basis,  in  March  2013,  we  began  working  with  FVAP  to  compare,  recon¬ 
cile,  and  align  what  was  in  the  agency’s  strategy  and  typical  of  its  operations  and  what 
should  be.  We  were  not  commissioned  to  conduct  a  program  evaluation  but  to  catalyze 
change. 

Our  tasking,  as  per  the  PD,  was  to  do  the  following: 

•  Assist  FVAP  in  aligning  its  strategy  and  operations  to  better  serve  its  mission. 

•  Strengthen  FVAP’s  capacity  to  set  its  own  course,  adapt  to  change,  and  commu¬ 
nicate  its  role  in  the  voting  community,  both  internally  and  externally. 

But,  it  became  clear  early  on  that  we  would  need  to  help  the  agency  address  the 
ambiguity  of  its  mission  and,  in  particular,  to  achieve  a  common,  shared  understand¬ 
ing  of  the  meaning  of  the  word  assistance. 

We  planned  the  project  for  four  phases,  including  two  analytical  phases  (see 
ph  ases  1  and  2,  Table  1.2).  We  front-loaded  the  analysis  to  enable  and  support  early 
action  by  the  agency,  but  the  distinctions  among  phases  became  less  consequential 
over  time.  As  the  two  organizations  began  to  work  more  closely,  the  “understanding, 
documentation,  and  assessment”  of  phase  1  begot  insight,  dialogue,  and  change  that 
continued  in  phase  2  and  beyond. 

The  process  was  not  always  easy  or  satisfying — at  times,  it  was  genuinely  frustrat¬ 
ing  for  everyone,  especially  when  people  from  the  two  organizations  were  using  words 
differently  or  at  cross  purposes — but  the  relationship  evolved,  and  with  that  evolution 
came  a  shared  vocabulary.  In  phase  1,  we  developed  a  deeper  understanding  of  FVAP, 
but  FVAP  was  also  getting  to  know  us — and  itself. 

Sequentially,  we 

•  developed  a  logic  model  to  benchmark  the  agency’s  strategy  and  operations 

•  engaged  with  FVAP’s  stakeholders,  consisting  of  congressional  staff,  election  offi¬ 
cials,  overseas-citizen  NGOs,  other  DoD  entities,  academics  and  technologists, 
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Table  1.2 
Project  Phases 


Phase 

Description 

Output 

Timeline 

Pre 

Preliminary  discussion  of  terms  of  engagement  and  PD 

PD 

March  2013 

1 

Understand,  document,  and  assess  FVAP's  strategy  and 
operations,  including  its  organizational  structure 

Preliminary  assessment, 
recommendations,  and 
guidance  (benchmark 
logic  model) 

Interim  briefings 

November 

2013a 

2 

Examine  how  FVAP  engages  with  and  responds  to  its 
operating  environment  through  partnerships,  research, 
and  other  activities 

Interim  briefings 
(forward-looking  logic 
models)13 

March 

2014c 

3 

Encapsulate  lessons  from  each  of  the  foregoing 
phases  to  provide  FVAP  with  a  clear  set  of  final 
recommendations  and  implementing  guidance  for 
making  the  changes  necessary  to  align  its  strategy, 
operations,  and  organization  and  to  identify  means  to 
better  communicate  its  role  in  the  voting  community, 
both  internally  and  externally 

Final  assessment, 
recommendations,  and 
guidance  (robust  logic 
model)b 

Final  briefings 

June  2014d 

4 

Disseminate  findings 

Reports  and 
stakeholder  meetings 

March 

2015e 

SOURCES:  RAND  Corporation,  2013,  and  additional  RAND  staff  analysis. 

a  For  phase  1,  the  timeline  specifies  the  delivery  of  the  preliminary  recommendations  and  initial 
guidance. 

b  The  forward-looking  and  robust  models  eventually  converged. 

c  For  phase  2,  the  timeline  specifies  the  approximate  date  of  the  completion  of  the  analysis. 
d  For  phase  3,  the  timeline  specifies  the  approximate  dates  of  the  final  briefings  to  FVAP. 
e  For  phase  4,  the  timeline  specifies  the  completion  of  the  draft  final  report. 


and  various  election  advocates  to  elicit  their  views  of  the  agency’s  role  in  the 
voting  community 

•  analyzed  the  agency’s  core  requirements  as  manifested  in  U.S.  law  and  policy  on 
uniformed-  and  overseas- citizen  absentee-voting  assistance. 

The  benchmark  model  served  as  a  diagnostic  tool  that  led  us  down  the  paths  of 
outreach  and  analysis  that  followed,  by  surfacing  issues  that  merited  closer  consid¬ 
eration.  The  modeling  effort,  which  included  a  two-day  all-hands  workshop,  weekly 
office  hours,  and  several  small-group  sessions,  enabled  us  to  synthesize  FVAP’s  under¬ 
standing  of  itself  and  to  chart  its  operations.  However,  it  also  confirmed  the  blurri¬ 
ness  of  FVAP’s  mission  and  suggested  holes  in  our  understanding  of  the  agency  and 
its  role  in  the  voting  community,  which  we  set  out  to  fill  with  stakeholder  outreach 
and  the  core  requirements  assessment.  Whereas  the  benchmark  logic  model  flowed 
from  our  discussions  with  FVAP  and  so  depicted  FVAP  as  FVAP  saw  itself,  the  stake¬ 
holder  engagement  and  requirements  assessment  allowed  us  to  see  FVAP  as  others  saw 
it.  When  we  compared  these  perspectives,  we  found  additional  evidence  of  gaps  and 
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disconnects,  which  we  discussed  with  FVAP,  but  we  also  began  to  identify  potential 
solutions. 

We  concluded  phase  1  with  the  formal  delivery  of  our  findings  and  preliminary 
recommendations,  but,  by  this  time,  FVAP  had  already  begun  working  with  our  sug¬ 
gestions,  including  those  relating  to  mission  clarification.  This  was  possible  because  we 
shared  our  findings  as  they  emerged  and  worked  with  FVAP,  in  close  communication 
throughout  the  process.  We  were  learning  what  we  needed  to  learn  with  each  task, 
reporting  what  we  found  as  we  found  it,  and  exploring  the  implications  with  FVAP. 
We  did  not  present  our  findings  as  fiat  but  as  fodder  to  generate  debate  and  discussion. 

In  phase  2,  we  took  a  closer  look  at  FVAP’s  engagement  with  its  operating  envi¬ 
ronment,  including  its  customers,  partners,  and  other  stakeholders;  related  organiza¬ 
tions;  and  supporting  infrastructure  and  technology,  from  the  perspective  of  a  larger 
voting  assistance  system.  We  paid  special  consideration  to  FVAP’s  work  with  mili¬ 
tary  installations,  including  the  IVA  offices,  and  the  selection  and  management  of 
research-related  activities.  During  this  phase,  the  practical  significance  and  implica¬ 
tions  of  the  collaborative  relationship — and  the  dynamism  of  the  agency’s  operating 
environment — became  apparent.  The  agency  was  changing  in  response  to  our  analysis, 
to  internally  recognized  needs,  and  to  exogenous  forces,  and  our  approach  to  the  analy¬ 
sis  was  also  changing. 

Two  developments  stand  out  as  having  shaped  both  FVAP  and  the  project  in 
phase  2.  The  first  was  the  development  and  refinement  of  the  robust  logic  model,  which 
set  the  stage  for  organizational  change,  and  the  second  was  the  promulgation  of  draft 
legislation,  known  as  the  Safeguarding  Elections  for  Our  Nation’s  Troops  Through 
Reforms  and  Improvements  Act  (S.  1728,  2013).  If  enacted,  the  proposed  legislation 
would  have  affected  FVAP’s  responsibilities  with  respect  to  research  and  the  delivery 
of  voting  assistance  to  uniformed  absentee  voters.  For  research,  the  act  would  have 
eliminated  the  requirement  for  an  electronic  voting  demonstration  project,  which  had 
been  a  focus  of  the  agency’s  research  efforts.10  Given  the  potential  for  change,  FVAP 
and  we  agreed  to  concentrate  less  on  particular  research-related  activities,  including 
the  demonstration  project,  and  the  configuration  of  installation  infrastructure,  which 
might  be  in  flux,  and  more  on  the  persistent  issues  that  surrounded  them.  We  looked 
at  the  effectiveness  of  the  agency’s  work  with  the  military  and  the  methods  the  agency 
was  using  to  formulate,  conduct,  and  use  research. 

During  this  period,  we  observed  substantial  change  within  FVAP.  Some  of  the 
change  stemmed  from  our  collaborative  efforts,  including  those  on  the  logic  model, 
and  some  originated  with  FVAP,  reflective  of  the  agency’s  interest  in  improvement. 


10  Subsequent  to  this  analysis,  Section  593  of  the  Carl  Levin  and  Howard  P.  “Buck”  McKeon  National  Defense 
Authorization  Act  for  Fiscal  Year  2015  (Pub.  L.  113-291,  2014),  titled  “Repeal  of  Electronic  Voting  Demonstra¬ 
tion  Project,”  eliminated  the  requirement. 
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In  broad  terms,  the  collaboration  went  as  planned.  We  delivered  recommenda¬ 
tions  and  implementing  guidance  to  provide  FVAP  with  the  means  to  more  effectively 
focus  its  strategy,  organize  and  execute  its  operations,  and  interact  proactively  and 
productively  with  stakeholders.  FVAP,  for  its  part,  worked  with  us  to  inform  the  pro¬ 
cess  and  suggest  tactical  changes  and  used  interim  findings  to  formulate  and  expedite 
change.  The  extent  of  the  collaboration  implied  a  substantial  investment  of  time  and 
focus  on  FVAP’s  part  (see  Table  1.3). 

Table  1.3 

Project  Events  and  Approximate  Federal  Voting  Assistance  Program  Staff  Commitments 


Description 


Participants 


Duration 


Preliminary  discussions 


Kickoff  meeting 

Logic-modeling  workshop 

Discussions  of  benchmark  logic 
model,  small-group  format 


Office  hours 


Discussions  of  preliminary 
recommendations  and  guidance, 
small-group  format 

Discussion  of  preliminary 
recommendations  and  guidance, 
large-group  format 

IVA  office  visits  and  training 
observations  at  nine 
installations3 

Discussions  of  final,  robust  logic 
model,  small-group  format 

Discussions  of  findings  on  FVAP's 
role  in  voting  assistance  system, 
small-group  format 

Discussion  of  findings  on  FVAP's 
role  in  voting  assistance  system, 
large-group  format 

VAO  training  workshop,  large- 
group  format 

VAO  training  workshop,  small- 
group  format 

VAO  training  workshop, 
feedback 


FVAP  leadership  with  input  from 
DHRA 


All  hands 
All  hands 

One  to  eight  FVAP  staff 
members,  including  leadership 


One  to  three  FVAP  staff 
members,  including  leadership 

One  to  three  FVAP  staff 
members,  including  leadership 


All  hands 


Multiple  one-  and  two-hour 
exchanges,  spread  over  several 
weeks 

Half-day  event 

Two-day  event 

Multiple  one-  and  two-hour 
exchanges,  spread  over  several 
weeks 

One  to  two  hours  per  week,  as 
needed 

Unable  to  estimate 


Two-hour  event 


One  or  two  FVAP  staff  members,  One-  to  three-day  events13 
including  leadership 

One  to  three  FVAP  staff  Unable  to  estimate 

members,  including  leadership 

One  or  two  FVAP  staff  members,  Unable  to  estimate 
including  leadership 


All  hands  Two-hour  event 


All  hands  Half-day  event 

Six  to  eight  FVAP  staff  members,  Two-hour  event 
including  leadership 

One  or  two  FVAP  staff  members,  Two-hour  event 
including  leadership 
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Table  1.3 — Continued 


Description 

Participants 

Duration 

Discussions  of  research-related 
and  other  project  activities, 
various  formats 

Unable  to  estimate 

Multiple  one-  and  two-hour 
exchanges,  spread  over  several 
weeks 

Discussion  of  methods  of  project 
management 

Eight  to  ten  FVAP  staff 
members,  including  leadership 

Two-hour  event 

Project-management  tool 
workshop 

All  hands 

Half-day  event 

Discussions  of  evidence  of 
change 

One  or  two  FVAP  staff  members, 
including  leadership 

Multiple  one-hour  exchanges, 
spread  over  several  weeks 

Ongoing  discussions  of 
approach,  findings,  and 
implications  throughout  process 

One  or  two  FVAP  staff  members, 
including  leadership 

Unable  to  estimate 

SOURCE:  RAND  staff  analysis. 

NOTE:  Participant  and  duration  estimates  are  rough  approximations. 
a  See  Table  6.1  and  the  discussion  in  Chapter  Six. 

b  FVAP  staff  would  have  conducted  VAO  training,  regardless  of  our  presence. 

In  phase  3,  we  synthesized  the  results  of  the  engagement  and  delivered  our  final 
recommendations  and  guidance.  Given  the  extent  of  the  change  that  had  already 
occurred,  we  directed  our  attention  to  locking  in  and  building  on  gains. 

In  this  project,  we  employed  a  standard,  time-tested,  and  readily  available 
method — namely,  logic  modeling — which  we  supplemented  with  stakeholder  out¬ 
reach,  a  requirements  assessment,  and  consideration  of  the  larger  voting  assistance 
system.  And  the  collaborative  relationship  made  it  possible  for  FVAP  to  make  changes 
expeditiously.  We  can  claim  robustness,  relevance,  and  timeliness,  but  not  uniqueness. 
Two  human  factors  also  stood  out  as  essential  to  progress:  the  agency’s  commitment  to 
implementing  change  from  the  outset  and  the  mutual  trust  that  an  ongoing,  collabora¬ 
tive  relationship  can  engender. 

Throughout  the  course  of  our  engagement,  another  RAND  project  team  was 
working  with  FVAP  to  develop  measures,  metrics,  and  a  dashboard  to  support  situ¬ 
ational  awareness  and  the  continuous  assessment  of  the  effectiveness  of  FVAP  activi¬ 
ties.  The  two  projects  took  complementary  approaches  to  strengthening  the  agency. 
Whereas  our  project  was  primarily  qualitative,  theirs  was  primarily  quantitative; 
whereas  ours  developed  the  logic  model,  theirs  used  the  model  to  identify  metrics  and 
measures  with  which  to  assess  progress. 
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Organization  of  This  Report 

In  the  remainder  of  the  report,  we  describe  the  project  in  terms  of  our  approach  to  the 
engagement,  a  set  of  outcomes,  and  the  process  that  connected  them.  First,  we  provide 
an  overview  of  the  two  analytical  phases  of  the  project,  i.e.,  “understanding,  docu¬ 
menting,  and  assessing”  and  “examining  FVAP’s  engagement,”  even  if  the  phases  were 
not  neatly  separable.  Second,  we  describe  changes  unfolding  within  FVAP,  some  in 
direct  response  to  our  ongoing  analysis  and  engagement  and  some  emerging  contem¬ 
poraneously  and  indicative  of  the  agency’s  desire  to  progress  rapidly. 

The  discussion  of  our  approach  unfolds  chronologically  in  narrative  form  over 
several  chapters.  It  is  our  intent  to  convey  the  experience  of  the  project  as  much  as  its 
outcomes  and,  thus,  to  provide  readers  with  insight  to  the  process  and  an  indication 
of  the  possible. 

In  addition  to  the  findings,  recommendations,  and  guidance  from  our  analysis, 
this  report  describes  in  detail  the  collaborative  process  that  FVAP  and  we  used  to 
develop  the  analysis  and  implement  recommendations,  respectively.  We  have  struc¬ 
tured  the  document  to  meet  the  needs  of  FVAP’s  leaders  as  they  convey  the  process 
that  they  used  to  implement  major  organizational  change  and  to  aid  the  agency  in  fur¬ 
ther  implementation  of  recommendations. 

The  report  proceeds  as  follows.  In  Chapters  Two,  Three,  and  Four,  we  discuss  the 
logic  modeling  exercise  and  its  implications,  our  engagement  with  FVAP’s  stakehold¬ 
ers,  and  FVAP’s  activities  in  relation  to  requirements,  respectively.  In  Chapter  Five, 
we  present  our  preliminary  recommendations  and  initial  guidance,  which  summarize 
our  findings  from  phase  1  and  provide  a  means  of  gauging  the  change  that  we  report 
in  Chapter  Seven.  In  Chapter  Six,  which  reports  on  phase  2  of  the  project,  we  take  a 
closer  look  at  FVAP’s  engagement  with  its  operating  environment,  identify  additional 
opportunities  to  improve  its  engagement,  and  present  the  final,  robust  logic  model.  In 
Chapter  Seven,  we  discuss  the  evidence  of  change  within  FVAP,  both  over  the  course 
of  the  project  and  in  its  final  phases.  In  Chapter  Eight,  we  present  recommendations  to 
lock  in  and  build  on  recent  gains  and  concluding  remarks  on  goals  and  results. 

In  Appendixes  A  through  F,  we  provide  a  detailed  description  of  the  logic¬ 
modeling  methodology;  the  laws  that  govern  FVAP;  survey  response  rates  and  factors 
that  affect  them;  the  project-management  tools  that  we  developed  for  FVAP;  our  anal¬ 
ysis  of  FVAP’s  VAO  training  program,  which  applies  those  tools;  and  adult  learning 
principles  and  training  evaluation. 


CHAPTER  TWO 


Logic-Model  Development 


As  the  starting  point  for  understanding  and  describing  FVAP’s  organizational  pri¬ 
orities  and  activities,  we  developed  a  benchmark  logic  model — an  exercise  that  we 
view  as  being  as  much  about  process  as  about  product.  For  the  purposes  of  phase  1  of 
this  project,  we  emphasized  the  role  of  the  modeling  process  and  its  contributions  to 
improving  our  understanding  of  FVAP,  the  activities  it  undertakes,  and  its  reasons  for 
undertaking  them. 

Because  the  benchmark  model  emerged  largely  from  a  series  of  conversations  with 
FVAP,  it  shed  light  on  how  FVAP  viewed  itself  at  the  start  of  the  project  and  provided 
us  with  a  starting  point  for  assessing  what  FVAP  was  doing  and  why  it  was  doing  it. 
The  model  helped  us  to  identify  discontinuities,  misalignments,  and  possible  ineffi¬ 
ciencies  in  the  agency’s  activities  in  relation  to  intended  outcomes  and,  thus,  served  as 
an  important  diagnostic  tool. 


Logic-Model  Basics 

Logic  models,  which  often  look  like  flow  charts,  can  be  used  to  articulate  the  critical 
path  by  which  a  program1  seeks  to  achieve  outcomes  and  attain  its  mission,  construct 
a  well-aligned  strategy  that  supports  its  mission,  communicate  internally  and  exter¬ 
nally,  and  conduct  program  evaluations.  Although  not  all  such  roles  were  relevant  to 
phase  1  of  this  project,  most  would  be  relevant  to  FVAP  in  other  contexts,  including 
later  phases  of  the  project.2 


1  This  discussion  draws  heavily  from  Greenfield,  Williams,  and  Eiseman,  2006;  Williams  et  al.,  2009;  and 
material  presented  during  a  two-day  logic-modeling  workshop,  held  at  FVAP.  For  more  on  the  use  of  logic 
models,  see  McLaughlin  and  Jordan,  1999,  and  Wholey,  Hatry,  and  Newcomer,  2010.  Appendix  A  provides  a 
set  of  notes  on  logic  modeling  that  we  left  with  workshop  participants.  The  notes  offer  a  fuller  discussion  of  logic 
modeling  and  suggest  additional  references  on  logic  modeling  and  applications.  The  concepts  of  logic  modeling 
apply  broadly  to  programs,  agencies,  and  other  institutional  configurations,  including  operational  lines  and  ini¬ 
tiatives;  here,  we  follow  the  literature  and  refer  to  programs. 

2  For  example,  in  Chapter  Eight,  we  discuss  the  model’s  applicability  to  communications. 
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A  logic  model  can  provide  a  simplified  visual  representation  of  a  program’s  opera¬ 
tions,  starting  with  inputs  and  progressing  to  activities,  outputs,  customers,  and  out¬ 
comes;  it  can  also  map  a  program’s  operations  to  its  strategy,  consisting  of  the  goals, 
objectives,  and  performance  measures  that  support  the  program’s  mission.3  In  addi¬ 
tion,  a  logic  model  can  be  used  to  draw  distinctions  between  partners  that  work  with  a 
program  to  create  outputs  and  customers  that  use  those  outputs,  either  immediately  or 
eventually.  In  some  instances,  the  same  entity  might  serve  as  a  partner  in  one  context 
and  as  a  customer  in  another. 

In  brief,  we  can  summarize  the  terminology  usage  as  follows: 

•  Inputs  are  resources  that  go  into  and  guide  program  activities. 

•  Activities  are  the  actions  that  the  program  undertakes  to  produce  outputs. 

•  Outputs  are  the  products  (goods  or  services)  that  activities  generate. 

•  Partners  are  those  that  work  with  programs  to  conduct  activities  or  enable  out¬ 
puts. 

•  Customers  (intermediate  and  final)  are  the  users  or  target  of  the  outputs. 

•  Outcomes  (intermediate  or  end)  are  the  changes  that  occur  and  the  benefits  that 
result  from  the  program  activities  and  outputs.  They  involve  changes  in  knowl¬ 
edge,  attitudes,  and  behaviors.  Intermediate  outcomes  reflect  the  customers’ 
response  to  the  program’s  outputs;  end  outcomes  are  the  desired  results  of  the 
program. 

•  Impact  is  a  program’s  contribution  to  a  societal  outcome. 

The  path  from  inputs  to  outcomes  might  not  be  strictly  linear  or  uninterrupted. 
Feedback  and  external  factors  (e.g.,  the  enactment  of  a  new  legislative  requirement, 
emergence  of  a  new  technology  or  security  concern,  or  change  within  a  partner  agency) 
can  affect  the  program’s  orientation  and  the  extent  to  which  it  can  achieve  outcomes. 

Figure  2.1  shows  the  logic-model  template  that  RAND  researchers  have  devel¬ 
oped  through  their  work  with  other  agencies  across  and  outside  the  federal  govern¬ 
ment.4  The  red  dotted  line,  positioned  beneath  customer  activities,  intermediate  goals, 
and  intermediate  measures,  signifies  a  point  along  the  path  at  which  a  program’s  con¬ 
trol  greatly  diminishes.5 

Notwithstanding  the  broad  applicability  of  logic  modeling — and  the  template  in 
Figure  2.1 — the  work  of  phase  1  focused  initially  on  FVAP’s  mission  and  operations. 


3  Operations  involve  resources,  actors,  and  events,  whereas  strategy  speaks  of  intentions. 

4  RAND  teams  have  used  this  template  to  assist  other  federal  agencies  (e.g.,  the  National  Institute  for  Occupa¬ 
tional  Safety  and  Health,  the  National  Center  for  Injury  Prevention  and  Control,  and  U.S.  Customs  and  Border 
Protection)  and  address  similar  issues.  For  documented  examples  of  each  type  of  use,  see  Greenfield,  Williams, 
and  Eiseman,  2006;  Greenfield,  Willis,  and  LaTourrette,  2012;  and  Williams  et  ah,  2009. 

5  For  additional  information  about  this  template  and  its  structure,  see  Appendix  A. 
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Figure  2.1 

Logic-Model  Template 
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SOURCE:  Greenfield,  Williams,  and  Eiseman,  2006. 

NOTE:  For  additional  information  about  this  template  and  its  structure,  see  Appendix  A. 
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The  Modeling  Process  Used  with  the  Federal  Voting  Assistance 
Program 

To  begin  developing  FVAP’s  benchmark  logic  model,  we  held  a  two-day  all-hands 
workshop  with  FVAP  staff  and  leadership  to  explore  the  agency’s  mission,  its  opera¬ 
tions,  and,  to  a  lesser  extent,  its  strategy.  To  spur  the  process,  we  provided  FVAP  with 
a  set  of  discussion  points  before  the  workshop.  The  discussion  points  addressed  three 
basic  logic-modeling  concepts — specifically,  who  are  you  as  a  program  or  program 
area,  what  are  you  doing,  and  why  are  you  doing  it?  In  the  workshop,  we  introduced 
the  logic  model,  framing  it  as  a  tool  for  better  aligning  program  strategy  and  opera¬ 
tions  and,  eventually,  for  strengthening  program  communication — and  FVAP  and  we 
put  the  tool  to  use.6 


6 


Appendix  A  includes  the  logic-modeling  primer,  the  agenda,  and  other  workshop  materials. 
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The  workshop’s  success  depended  not  on  whether  it  yielded  a  pretty  picture  but 
on  whether  the  staff — and  our  RAND  project  team — walked  away  with  a  clearer, 
common  understanding  of  the  relationships  among  the  things  FYAP  was  doing  as  an 
institution  and  why  it  was  doing  them.  By  the  end  of  the  workshop,  we  did  not  expect 
to  have  a  fully  formed  logic  model — or  a  well-articulated  “road  ahead” — but  to  be  on 
the  way  to  having  both. 

Typically,  an  agency’s  mission  statement  would  serve  as  a  foundation  for  this 
type  of  logic  model;  however,  in  this  case,  a  day  1  workshop  session  titled,  “Why  Does 
FVAP  Exist?”  confirmed  that  FVAP  staff  members  lacked  a  common,  shared  under¬ 
standing  of  the  organization’s  purpose.  We  consider  the  implications  of  that  finding 
below,  but,  to  fill  the  immediate  need  for  a  foundation,  we  proposed,  discussed,  and 
adopted  a  rough  proxy:  to  assist  UOCAVA  voters  in  voting  successfully. 

With  that  proxy  in  hand,  we  were  able  to  work  with  FYAP — during  the  work¬ 
shop  and  in  subsequent  meetings — to  chart  the  agency’s  operations  and  develop  an  ini¬ 
tial  logic  model  that  constituted  a  programmatic  benchmark.  It  was  reflective  of  then- 
current  practice,  including  some  recent  changes  in  practice,  and  anticipated  future 
needs. 

For  example,  in  a  day  2  workshop  session,  we  asked  FVAP  staff  to  identify  inputs, 
activities,  outputs,  customers,  and  outcomes  and  consider  how  the  proposed  elements 
would  support  the  proxy  mission.  Staff  wrote  down  their  inputs,  activities,  outputs, 
outcomes,  partners,  and  customers  on  sheets  of  paper  and  taped  them  to  a  room- 
length  white  board,  under  the  appropriate  category  headings.  The  snapshot  shown  in 
Figure  2.2,  which  we  cropped  to  fit  this  document,  shows  only  inputs,  activities,  and 
outputs,  but  the  board  continued  several  feet  to  the  right  and  included  information  on 
customers  and  outcomes,  with  voters  and  successful  voting  featured  prominently. 

The  development  of  the  benchmark  logic  model,  which  occurred  over  a  period  of 
about  four  months,  exemplified  the  collaborative  and  iterative  approach  of  the  larger 
project.  We  took  our  gleanings  from  the  workshop  and  used  them  to  create  a  rough 
mock-up  of  the  benchmark  model.  Then,  we  met  with  members  of  the  FVAP  staff,  pri¬ 
marily  in  small  groups,  to  obtain  feedback  on  elements  of  the  model  and,  eventually, 
on  the  model  in  its  totality.  We  met  with  FVAP  to  discuss  the  model,  then  revised  the 
model,  met  with  FVAP  again,  and  so  forth.  The  model  was  complete  when  participants 
reached  agreement  on  the  content  and  structure. 

In  large  part,  the  development  of  the  benchmark  logic  model  was  an  act  of  syn¬ 
thesis.  FVAP  staff  members  could  see  themselves  within  the  organization  and  could 
describe  daily  tasks  and  projects  in  terms  of  activities  and  outputs  but  could  not  read¬ 
ily  see  themselves  as  part  of  a  larger  organization.  The  modeling  exercise  helped  us  and 
them  to  see  how  the  parts  might — or  might  not — fit  together  as  a  whole. 

To  provide  context  for  our  discussions  with  FVAP  and  enhance  our  understand¬ 
ing  of  the  organization,  we  also  embarked  on  a  complementary  process  of  reviewing 


Figure  2.2 

Constructing  the  Logic  Model 


SOURCE:  RAND  staff  photograph  taken  May  2013. 
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strategic  documents,  congressional  testimony,  training  materials  and  guidance,  surveys 
and  reports,  organizational  charts,  and  governing  legislation  and  policy.7 


The  Federal  Voting  Assistance  Program  Benchmark  Logic  Model 

Figure  2.3,  “The  Federal  Voting  Assistance  Program  Benchmark  Logic  Model,”  depicts 
the  results  of  the  modeling  workshop,  meetings  with  FVAP  staff,  and  the  document 
review. 

The  model  depicts  three  distinct  streams  of  activity:  voter  assistance;  election- 
official  assistance;  and  institutional  support,  including  communication  and  technical 
services.  The  streams  face  inward  (in  that  they  provide  service  to  the  agency  itself,  as  in 
the  case  of  program  oversight)  and  outward  (in  that  they  provide  service  to  others,  as 
in  the  case  of  outreach).  The  content  of  the  voter  and  election-official  assistance  activity 
boxes  is  quite  similar — for  example,  both  streams  develop  and  modify  training  materi¬ 
als,  guidance,  and  other  online  materials  and  build  and  maintain  relationships — even 
if  oriented  toward  different  audiences.  Separately  and  in  sum,  the  three  activity  streams 
yield  outputs,  which  are  then  transferred  to  groups  of  intermediate  customers,  such  as 
the  military  services,  the  Department  of  State,  states,  localities,  DOJ,  and  NGOs,  for 
their  use  and  to  the  ultimate  benefit  of  UOCAVA  voters,  framed  in  terms  of  votes  cast, 
received,  and  counted. 

The  logic  model  includes  a  category  of  activities  that  we  designated  as  special  proj¬ 
ects,  consisting  of  one-off  or  periodic  events  outside  the  realm  of  routine  day-to-day 
activities.  In  this  category,  we  included  research  undertaken  in  accordance  with  law, 
policy,  and  emerging  needs  and  revisions  to  forms,  instructions,  and  directives.  For 
example,  FVAP,  on  behalf  of  the  presidential  designee  and  serving  as  the  administra¬ 
tor  of  UOCAVA  for  USD(P&R),  holds  the  pen  in  the  process  of  revising  and  reissuing 
DoDI  1000.04,  which  is  the  primary  (DoD)  policy  document  concerning  UOCAVA 
voting,  roughly  every  two  years.8  On  that  basis,  one  might  view  the  DoDI  as  dual- 
hatted — upon  issuance,  it  is  an  external  factor,  but,  in  the  course  of  revisions,  its  coor¬ 
dination  and  authorship  are  activities. 

In  some  instances,  such  as  the  provision  of  online  assistants,9  call  responses,  email 
blasts  or  bulk  email,  and  other  media,  the  activity  streams  also  reach  directly  to  voters. 
For  example,  a  UOCAVA  voter  can  use  the  online  assistants  to  obtain  information 


7  For  this  purpose,  we  did  not  closely  examine  the  law  or  policy,  as  we  do  in  the  later  discussion  of  core  require¬ 
ments;  rather,  we  read  it  as  a  backdrop  to  our  conversations  with  FVAP  and  the  model-development  process. 

8  FVAP  acts  on  behalf  of  USD(P&R),  who  issues  the  guidance.  See  Chapters  Four  and  Seven  of  this  report 
for  a  discussion  of  the  process  and  DoDD  5124.02  (Director  of  Administration  and  Management,  2008)  for  an 
explanation  of  the  delegation  of  responsibilities. 

9  Online  assistant  refers  to  specific  tools  that  FVAP  directly  provides  and  supports  through  its  website. 
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SOURCE:  RAND  staff  analysis  with  input  from  FVAP  leadership  and  staff. 

NOTE:  DOS  =  U.S.  Department  of  State.  IG  =  inspector  general.  GAO  =  U.S.  Government  Accountability  Office.  SME  =  subject-matter  expert.  HR  =  human  resources.  ProfDev  = 
professional  development.  IT  =  information  technology.  VAG  =  Voting  Assistance  Guide.  PSA  =  public  service  announcement.  LMS  =  learning  management  system.  Wizard  = 
web-based  tool  that  voters  can  use  to  obtain  information  and  walk  through  the  process  of  completing  the  FPCA  and  FWAB,  which  are  now  referred  to  as  "online  assistants." 
SVAO  =  Service  Voting  Action  Officer.  OSD/P&R  =  USD(P&R).  PA  =  public  affairs.  LA  =  legislative  affairs.  DMDC  =  Defense  Manpower  Data  Center.  DTIC  =  Defense  Technical 
Information  Center.  MPSA  =  Military  Postal  Service  Agency.  WHS  =  Washington  Headquarters  Services.  DOJ  =  U.S.  Department  of  Justice.  NIST  =  National  Institute  of  Standards 
and  Technology.  EAC  =  U.S.  Election  Assistance  Commission.  USPS  =  U.S.  Postal  Service.  OMB  =  Office  of  Management  and  Budget.  OPM  =  Office  of  Personnel  Management. 
GSA  =  General  Services  Administration. 
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about  voting  in  his  or  her  jurisdiction  and  to  walk  through  the  process  of  complet¬ 
ing  the  Federal  Post  Card  Application  (FPCA)  and  Federal  Write-In  Absentee  Ballot 
(FWAB).  The  first  form,  the  FPCA,  allows  an  eligible  U.S.  citizen,  as  a  UOCAVA 
voter,  to  apply  to  register  to  vote,  request  an  absentee  ballot,  or  update  his  or  her 
contact  information  with  his  or  her  local  election  office.  The  FWAB  can  be  used  as 
a  backup  for  a  UOCAVA  voter  who  has  applied  for  a  regular  state  ballot  through  the 
FPCA  and  has  not  received  that  ballot  in  time  to  vote  by  the  election  deadline. 

The  large,  central  “transfer”  arrow  reaches  the  entire  box  of  customers  (on  the 
right  of  the  figure),  including  UOCAVA  voters.  As  a  practical  matter,  FVAP  has  more 
difficulty  reaching  out  directly  to  overseas-citizen  absentee  voters  than  to  uniformed 
absentee  voters.10 

Each  of  the  activity  and  corresponding  output  boxes  also  includes  the  words, 
“respond  to  information  and  service  requests”  and  “direct  responses,”  respectively.  The 
boxes  depict  not  just  flows  of  requests  for  information  about  voting  assistance  but  also 
flows  of  requests  for  other  types  of  information,  largely  involving  congressional  letters, 
press  reports,  and  other  media  activity,  which  appeared  to  be  substantial. 

The  depiction  in  Figure  2.3  weds  the  agency’s  activity  streams  and  its  organi¬ 
zational  structure  (see  Figure  2.4)  and,  on  that  basis,  could  be  considered  part  logic 
model  and  part  organizational  chart.  At  the  time  of  the  phase  1  analysis,  FVAP  housed 
17  permanent  FTE* 11  and  six  term  employees,  including  the  director  among  the  FTEs. 
The  staff  was  spread  across  five  organizational  groups,  consisting  of  voter  assistance, 
election-official  assistance,  technology,  communications,  and  mission  support — the 
last  three,  in  combination,  providing  various  types  of  institutional  support.  Three  sep¬ 
arate  deputies  each  led  voter  assistance,  election-official  assistance,  and  technology, 
respectively,  and  a  chief  of  staff  led  mission  support;  the  chief  of  staff  and  front  office 
jointly  led  the  communication  group.  The  hybridization  of  the  logic  model  offered  two 
benefits.  First,  it  made  it  easier  to  track  activities  that  are  similar,  such  as  relationship¬ 
building  and  maintenance,  but  directed  at  different  audiences  with  potentially  differ¬ 
ent  needs.  Second,  it  enabled  us  to  draw  out  potential  organizational  concerns,  relating 
to  the  positioning  of  the  boxes  and  the  connections  among  them. 

We  next  offer  a  final  comment  on  the  interpretation  of  the  model. 

The  benchmark  logic  model  was  based  largely  on  FVAP’s  sense  of  itself,  as  con¬ 
veyed  both  verbally,  through  the  workshop  and  later  discussions  with  staff,  and  in 
writing,  through  strategic  documents.  Thus,  it  represents  the  agency’s  theory  of  its 
operations,  absent  testing  and  validation.  Although  much  of  the  subsequent  work  of 
phase  1,  including  stakeholder  outreach,  spoke  at  least  notionally  to  testing  and  valida- 


10  Although  many  online  resources  are  available  to  all  UOCAVA  voters,  regardless  of  their  affiliation,  FVAP  has 
less  visibility  for  overseas  voters  who  are  not  members  of  the  uniformed  services,  less  direct  access  to  them,  and 
fewer  mechanisms  with  which  to  reach  out  to  them. 

11  For  tallying  purposes,  we  treated  one  full-time  DHRA  detailee  as  a  permanent  FTE. 


Figure  2.4 

Federal  Voting  Assistance  Program  Organization  Chart  as  of  April-May  2013 
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SOURCE:  RAND  staff,  adapted  from  chart  provided  by  FVAP  leadership. 

NOTE:  The  line  between  Mission  Support  and  Communications  indicates  that  Communications,  which  had  a  direct  line  to  the  director,  was  also  run 
as  part  of  Mission  Support.  For  some  purposes,  it  was  its  own  group;  for  others,  it  was  part  of  Mission  Support. 
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tion,  the  model  does  not  reflect  it.  For  that  reason,  the  arrows  connecting  the  various 
boxes  might  be  thick  or  thin,  and  the  reported  activities,  outputs,  and  outcomes  in 
those  boxes  might  be  real  or  aspirational.12 


Focal  Points  of  the  Logic-Model  Analysis 

In  this  section,  we  present  a  set  of  focal  points  that  emerged  from  the  logic-modeling 
process.  Figure  2.5  shows  that  the  model,  in  stylized  terms,  breaks  down  into  three 
distinct  activity  streams,  each  ultimately  reaching  out  to  the  UOCAYA  voter  as  the 
end  customer,  but  largely  dependent  on  the  actions  of  partners,  various  intermediate 
customers,  and  other  stakeholders.  Strictly  on  the  basis  of  that  image,  one  might  ask 
whether  the  activity  streams  are  well-integrated  and  reinforcing,  whether  FVAP  inter¬ 
faces  effectively  with  its  stakeholders,  and,  if  so,  to  what  ultimate  effect.  These  ques¬ 
tions  provide  a  backdrop  for  the  focal  points  that  emerged  from  the  analysis: 

•  mission  ambiguity 

•  stovepiping  and  fragmentation 

•  inadequate  capacities  and  capabilities  in  some  areas 

•  organizing  and  allocating  staff  resources. 

We  discuss  each  of  these,  in  turn,  below. 

Although  we  offer  comments  on  VAO  training  and  research-related  activities  in 
this  section,  we  note  that  we  intended  to  delve  more  deeply  into  both  arenas  in  phase  2 
of  this  project.  On  that  basis,  our  comments  on  these  issues  were  in  progress  during 
phase  1. 

Mission  Ambiguity 

Our  engagement  with  FYAP  and  exploration  of  strategy  documents  did  not  provide 
evidence  of  a  clearly  articulated,  commonly  understood,  or  shared  mission,  which, 
in  turn,  suggested  an  organizational  challenge.  Given  the  aforementioned  turnover 
in  leadership,  this  was  not  surprising  but,  nevertheless,  merited  comment.  If  FVAP 
staff  members  did  not  share  a  consistent  understanding  of  the  agency’s  mission — and 
underlying  priorities — they  could  not  be  expected  to  share  a  consistent  view  of  what 
the  agency — or  they — should  be  doing. 

The  draft  mission  statement,  shown  in  Table  1.1  in  Chapter  One,  and  the 
workshop-generated  proxy  suggested  agency-wide  convergence  on  a  common  end 
point  but  belied  disagreement  on  four  points:  first,  the  meaning  of  assist-,  second,  the 
meaning  of  successfully,  third,  the  position  of  the  voter  relative  to  other  stakeholders; 


12  In  our  discussions  with  FVAP,  we  acknowledged  that  the  agency  was  already  undergoing  a  certain  amount  of 
change  and  attempted  to  distinguish  between  what  is  and  what  is  to  be. 
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and,  fourth,  FVAP’s  stake  either  in  promoting  awareness  of  the  right  to  vote — and  the 
means  to  do  so — or  in  motivating  voting.13  Differences  of  opinion  about  assistance, 
success,  and  the  positioning  of  stakeholders  surfaced  during  the  workshop  and  became 
more  apparent  as  the  modeling  process  unfolded;  differences  in  views  regarding  FVAP’s 
stake  in  awareness  and  motivation  manifested  in  FVAP  documents,  including  the  IVA 
office  handbook  (FVAP,  undated  [a])  and  Voting  Assistance  Guide  (FVAP,  201 3f). 

We  use  the  fourth  point — on  promoting  awareness  or  motivation — to  draw  out  a 
few  of  the  consequences  of  mission  ambiguity  for  strategy  and  operations. 

Throughout  our  meetings  with  FVAP,  staff  signaled  that  the  agency  was  not  seek¬ 
ing  to  motivate  voters  but  to  inform  them.  Nevertheless,  training  materials  and  guid¬ 
ance  conveyed  the  impression,  if  only  to  the  layperson,  that  FVAP  might  have  been 
positioning  itself  as  a  motivating  agency,  e.g.,  in  citing  the  mission  of  FVAP  as  to 


13  To  distinguish  the  two  concepts,  we  treat  awareness  as  referring  to  UOCAVA  voters’  knowledge  of  basic  facts 
about  their  circumstances  and  opportunities  and  motivation  as  concerning  UOCAVA  voters’  levels  of  interest  in 
participating  in  the  voting  process.  Voter  motivation  is  sometimes  associated  with  the  phrase  “get  out  the  vote,” 
but  we  have  curbed  our  use  of  that  phrase  because,  for  some  readers,  it  might  conjure  images  of  partisan  canvass¬ 
ing  efforts. 
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“foster  voting  participation”  (FVAP,  undated  [a],  p.  4) 14  and,  more  explicitly,  in  spon¬ 
soring  and  distributing  “motivational”  posters.15 

FVAP  might  have  thought  itself  clear  on  its  position  regarding  awareness  and 
motivation,  but,  in  practice,  the  line  separating  them  was  blurry,  and  FVAP’s  public 
face — e.g.,  its  documents  and  other  resources — appeared  to  be  addressing  both.  The 
appearance  of  serving  to  raise  awareness  and  motivate,  even  absent  the  reality,  could 
bear  on  the  agency’s  accountability  and,  eventually,  the  allocation  of  resources  to  and 
across  the  agency.  If,  for  example,  FVAP  were  seen  as  a  motivator  and  not  an  informa¬ 
tion  provider,  its  stake  in  voter  participation  could  be  treated  as  substantially  greater, 
and  it  might  be  held  more  accountable  for  that  participation.16 

In  summary,  FVAP  would  have  difficulty  plotting  a  course  to  an  end  point  or 
intended  outcome  and  communicating  its  role,  lacking  a  shared  understanding  of 
purpose — hence  the  need  for  institutional  clarity  among  staff  members  and  across 
product  lines.  However,  it  would  not  be  enough  to  reach  agreement  on  words  alone.  It 
would  be  necessary  to  reach  agreement  on  the  meaning  of  the  words  and  the  priorities 
they  embody. 

The  modeling  process  also  revealed  concerns  within  FVAP  about  mission  creep. 
We  raise  that  issue  here,  in  relation  to  mission  ambiguity,  but  defer  further  consider¬ 
ation  of  scope  to  the  later  assessment  of  core  legal  requirements  in  Chapter  Four. 

Stovepiping  and  Fragmentation 

The  picture — both  literal  and  figurative — of  FVAP  that  emerged  from  the  model¬ 
ing  process  strongly  suggested  that  FVAP  was  not  functioning  as  a  program,  per  se, 
but  as  loosely  connected,  separately  managed  streams  of  activities.  That  stovepiping 
might  have  been  intentional  at  some  point  in  the  agency’s  history,  but  it  had  resulted 
in  fragmentation. 

The  benchmark  logic  model  reflects  this  disconnectedness  in  its  depiction  of  the 
voter  and  election-official  assistance  activity  streams.  The  two  streams  are  positioned  at 
a  distance,  but  absent  any  direct  ties.17  Nevertheless,  the  activities  of  the  streams  were 
strikingly  similar,  even  if  directed  at  different  intermediate  customers  or  audiences, 
thus  inviting  the  question  of  whether  more  could  be  done  to  leverage  FVAP  resources 
across  streams. 


14  This  publication,  which  is  the  IVA  office  handbook,  appeared  to  have  been  updated  after  September  2012, 
because  it  contained  references  to  material  published  in  that  month. 

15  At  that  time,  the  posters  had  their  own  tab  on  the  FVAP  website,  labeled  as  “Motivational  Posters”  under  the 
“Voting  Assistance  Officers”  heading. 

16  The  law  requires  FVAP  to  report  on  voter  participation  (see  the  later  discussion  of  statutory  requirements),  but 
FVAP  is  not  responsible  for  participation.  If  FVAP  were  to  take  on  motivation  explicitly,  the  public  could  reason¬ 
ably  hold  the  agency  responsible  for  participation. 

1  Had  we  drawn  the  streams  in  parallel,  stacked  one  atop  the  other,  the  ties  would  still  have  been  missing 
because  they  were  acting  more  or  less  independently  of  each  other. 
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The  following  highlight  two  examples  of  repetition: 

•  The  words  “build  and  maintain  relationships”  appeared  in  three  different  activ¬ 
ity  streams,  i.e.,  voter  and  election-official  assistance  and  communications.  In 
all  three  instances,  FVAP  staff  described  the  intent  of  relationship-building 
and  maintenance  as  engendering  access,  engagement,  and  cooperation.  Having 
established  this  similarity  of  purpose,  it  seemed  reasonable  to  ask  whether  FVAP 
employed  a  common  strategy  or  shared  methods  for  approaching  this  activity.18 

•  The  words  “develop,”  “modify,”  and  “identify,”  tied  to  training  materials, 
guidance,  other  online  content,  and  best  practices,  also  appeared  across  activ¬ 
ity  streams.  As  above,  the  repetition  raised  conceptual  and  technical  questions 
regarding  purpose  and  means.  Among  the  questions  were  whether  the  underly¬ 
ing  methods  of  development,  modification,  and  identification  were  empirically  or 
analytically  based.  Arrows  coming  from  other  boxes,  e.g.,  “inputs”  and  “institu¬ 
tional  support,”  allowed  the  possibility,  but  our  conversations  with  FVAP  staff  did 
not  confirm  the  reality. 

Although  depicted  as  being  more  connected  to  the  whole,  the  communication 
and  technical  service  groups — positioned  between  and  interacting  with  the  voter  and 
election-official  assistance  streams  in  the  logic  model — still  seemed  to  function  on 
their  own  in  many  regards.  To  illustrate,  we  explore  issues  involving  research-related 
activities. 

•  FVAP’s  research-related  activities,  led  primarily  through  the  technical  service 
group,  did  not  appear  to  provide  feedback  consistently  to  the  rest  of  the  agency. 
The  results  of  these  activities  were  intended  to  drive  up  to  leadership  and  then 
back  down  through  the  organization,  but  the  process  was  ineffective.  The  results 
of  surveys,  studies,  and  other  data  collection  might  have  filtered  through  the 
agency  eventually,  but  we  found  no  evidence  that  they  systematically  provided 
feedback  or  were  incorporated  regularly  into  the  voter  and  election-official  assis¬ 
tance  groups’  planning  and  operations.  A  lack  of  analytical  capacity  and  capabil¬ 
ity  might  be  an  explanatory  factor  (see  below);  in  short,  it  might  be  difficult  to 
create  feedback  loops  absent  the  analytical  foundation  to  harvest  new  informa¬ 
tion  or  assess  the  underlying  informational  needs.  Moreover,  it  appeared  that 
much  of  the  agency’s  research  had  been  undertaken  without  explicit  regard  for  its 
implications  for  FVAP’s  other  operationally  oriented  activity  streams. 


18  More  concretely,  if  FVAP  were  to  track  stakeholder  engagement  across  activity  streams  and  construct  an  accu¬ 
mulated,  consistent  history,  it  might  aid  the  identification  of  crosscutting  issues,  tracking  of  patterns  of  emer¬ 
gence,  and  development  of  unified  agency-wide  responses.  And,  it  might  better  enable  staff  based  in  one  activity 
stream  to  assist  staff  based  in  another. 
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•  Although  the  communication  service  group  carried  responsibility  for  messag¬ 
ing,  the  group  appeared  to  have  been  less  actively  involved  in  developing  voting 
and  election-official  assistance  guidance  and  training  materials,  including  the 
IVA  office  handbook,  the  Voting  Assistance  Guide,  and  workshops,  all  of  which 
contribute  to  the  public  face  of  FVAP.  If  the  agency’s  activities  were  more  fully 
integrated,  materials  speaking  to  mission  or  policy — regardless  of  their  origin  or 
purpose — could  be  honed  for  consistency  before  dissemination. 

Because  our  consideration  of  FVAP’s  research-related  efforts  were  to  continue  in  the 
next  phase  of  the  project,  we  deemed  these  immediate  observations  to  be  in  progress. 

inadequate  Capacities  and  Capabilities  in  Some  Areas 

Our  conversations  with  FVAP  and  review  of  strategic  and  other  documents  suggested 
that  FVAP  was  lacking  capacity  and  capabilities  in  some  organizationally  important 
areas.  These  areas  might  or  might  not  remain  organizationally  important  in  the  future, 
but  they  appeared  to  be  important  to  FVAP  in  its  then-current  form. 

One  of  FVAP’s  challenges,  as  noted  above,  was  an  apparent  inability  to  effectively 
harvest  new  information  or  ascertain  the  necessity  of  such  information.  To  illustrate, 
we  considered  the  postelection  surveys  that  FVAP  undertakes  and  through  which  it 
has  reached  out  separately  to  active-duty  military  (ADM)  personnel,  ADM  spouses, 
overseas  citizens,  unit  VAOs  (UVAOs),  State  Department  VAOs,  and  local  election 
officials.19  Our  analysis  of  the  citations  to  these  surveys  in  the  most  recent  postelection 
report  to  Congress  available  in  our  period  of  analysis  (FVAP,  2013b),  suggested  that 
only  a  small  fraction  of  the  information  obtained  from  the  surveys  was  put  to  use  in 
reporting.20  (See  Table  2.1.)  Regarding  the  necessity  of  the  surveys,  the  law,  as  we  dis¬ 
cuss  in  a  later  chapter,  requires  that  FVAP  report  on  a  limited  set  of  concerns,  for  exam¬ 
ple,  regarding  the  effectiveness  of  activities  and  utilization  of  certain  procedures  and  a 
statistical  analysis  of  voter  participation,  but  does  not  specify  a  method  of  data  collec¬ 
tion.  Although  the  surveys  have  provided  a  basis  for  the  statistical  analysis  and  might 
have  added  value  to  management,  the  extent  of  their  use  merited  further  investigation. 

Moreover,  in  some  cases,  it  was  unclear  whether  other  research-related  activities, 
such  as  some  of  the  discrete  special  projects  featured  in  the  benchmark  logic  model, 
were  structured  to  help  FVAP  achieve  end  outcomes.  We  suggested  two  possible  imped¬ 
iments.  First,  FVAP  might  have  difficulty  framing  clear,  mission-directed  research 
questions,  which  is,  in  itself,  an  analytical  undertaking  that  requires  a  not-insubstantial 
capacity  and  capability.  Second,  research-related  activities  had  not  been  routinely  con¬ 
ceptualized  as  activities  undertaken  to  support  other  FVAP  activity  streams. 


19  FVAP  and  EAC  were  in  the  process  of  merging  their  respective  quantitative  surveys. 

20  FVAP  notes  that  this  was  an  active  decision  by  FVAP,  based  on  two  key  factors:  precedent  through  previous 
reports  and  overall  readability. 
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Table  2.1 

Questions  Cited  in  the  2012  Postelection  Report  to  Congress 


Survey 

Number  of  Questions 
Cited  in  Report 

Total  Number  of 
Questions  Asked  in 
Survey 

Percentage  of  Total 
Questions  Cited  in  Report 

ADM 

21 

79 

27 

Local  election  officials 
(quantitative) 

27 

102 

26 

Local  election  officials 
(qualitative) 

5 

26 

19 

ADM  spouses 

9 

53 

17 

Department  of  State 
VAOs 

5 

34 

15 

DoD  UVAOs 

6 

42 

14 

SOURCES:  RAND  staff  analysis  based  on  FVAP,  2013b. 

NOTE:  We  list  the  surveys  in  descending  order  by  percentage  of  total  questions  cited  in  the  report. 


Although  the  discussion  in  this  section  focused  on  research  capacities  and  capa¬ 
bilities,  deficits  might  have  been  at  play  elsewhere,  including  in  the  voter  and  election- 
official  assistance  streams.  Although  professional  development  is  present  in  the  logic 
model,  we  heard  little  in  our  conversations  to  suggest  that  it  was  a  high  priority  within 
the  agency  or  well-targeted.  Training  for  staff  had  been  available  to  some  upon  request, 
but  we  were  unaware  of  systematic  processes  for  discerning  or  meeting  needs.  More¬ 
over,  we  heard  no  mention  of  cross-training  across  activity  streams.  It  did  not  appear 
that  FVAP  staff  in  one  activity  stream  received  explicit  training  to  gap-fill  or  trouble¬ 
shoot  in  other  streams.21 

Staffing  and  Organization  in  Relation  to  Mission 

The  final  set  of  issues  emerged  partly  from  the  drafting  of  the  logic  model,  but  more 
directly  from  our  conversations  with  staff  about  the  model.  These  issues  pertain  to  the 
ways  in  which  FVAP  was  conceptualizing  labor  and  organizing  itself,  both  in  general 
and  in  the  face  of  a  labor  constraint  of  fewer  than  20  permanent  FTE  positions. 

FVAP’s  staff  members — like  any  other  agency’s — are  not  free  resources,  nor,  in 
an  FTE-constrained  workplace,  would  the  costs  of  their  compensation  packages  nec¬ 
essarily  reflect  their  true  value  to  the  organization.  Whereas  FVAP  had  filled  some  of 
its  needs  through  term  appointments22  and  outside  contracts,  it  was  not  obvious  that 


21  We  later  learned  that  that  election-official  assistance  staff  had  begun  to  attend  voting  assistance  training  ses- 
sions  and  viewed  this  as  a  step  in  the  direction  of  cross-training. 

22  Some  of  these  arrangements  might  merit  further  consideration.  For  example,  if  a  role  is  or  becomes  lasting,  it 
might  require  a  permanent  assignment. 
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the  agency  had  always  recognized  the  implications  of  a  labor  constraint.  If  staffing  at 
FVAP  were  zero-sum,  then  a  decision  to  use  staff  in  one  activity  stream  would  con¬ 
stitute  a  decision  to  not  use  staff  in  another,  and  the  allocation  decision  would  either 
establish  or  act  on  the  agency’s  priorities. 

Moreover,  FVAP’s  organizational  structure  appeared  to  be  top  heavy  and,  per¬ 
haps,  support  heavy.  At  the  time  of  the  phase  1  analysis,  about  one-third  of  FVAP 
staff — the  director,  five  other  permanent  FTEs,  and  one  term  position — served  in 
leadership  or  advisory  capacities.  An  also-large  fraction  of  the  agency  appeared  to  be 
dedicated  to  support-like  functions,  possibly  reflecting  the  characterization  of  their 
roles  more  than  their  functions.  Including  leadership  and  advisory  positions  other  than 
the  director,  FVAP  allocated  six  permanent  FTEs  and  three  term  employees  to  voter 
and  election-official  assistance  and  ten  permanent  FTEs  and  three  term  employees  to 
positions  that  might  have  been  categorized  as  support,  depending  especially  on  the 
underlying  nature  of  communication  and  technology  services. 


Concluding  Remarks  on  Logic-Model  Development  and  Findings 

In  this  chapter,  we  demonstrated  ways  in  which  the  logic  model  can  be  used  to  better 
understand  an  organization,  diagnose  problems  within  an  organization,  and  begin  to 
identify  potential  solutions.  Although  the  specifics  of  the  modeling  exercise  and  the 
resulting  diagram  will  differ  from  organization  to  organization,  the  method  itself  can 
be  applied  across  settings  and  agencies  for  similar  purposes. 

At  the  outset  of  the  discussion  of  focal  points,  we  noted  that  the  benchmark  logic 
model  raised  questions  about  whether  FVAP’s  activity  streams  were  well-integrated 
and  reinforcing,  whether  FVAP  interfaced  effectively  with  its  stakeholders,  and,  if  so, 
to  what  end.  Additional  analysis  of  the  model  suggested  that  FVAP  activities  were  not 
well-integrated  or  reinforcing,  but  it  left  open  the  questions  of  the  effectiveness  and 
outcomes. 

Through  the  development  of  the  logic  model,  we  became  aware  of  holes  in  our 
understanding  of  the  agency  and  its  role  in  the  voting  community.  FVAP  was  under¬ 
taking  various  activities  from  which  it  was  creating  outputs,  but  then  what  happened 
to  them?  Where  did  the  outputs  go,  how  were  they  used,  and  to  what  ultimate  effect? 
To  begin  to  complete  the  picture,  our  next  step  was  to  approach  FVAP’s  stakeholders, 
particularly  its  partners  and  intermediate  customers,  for  insight.  In  so  doing,  we  would 
also  be  able  to  test  and  validate  what  we  believed  we  already  understood  about  the 
agency — as  depicted  in  the  benchmark  logic  model — and  explore  the  effectiveness  and 
value  of  the  stakeholder  interface. 


CHAPTER  THREE 


Stakeholder  Outreach 


In  this  chapter,  we  discuss  our  approach  to  stakeholder  outreach  and  the  key  insights 
that  we  drew  from  it.  It  was  our  intent  to  learn  more  about  the  successes  and  failures 
of  the  relationships  between  FVAP  and  its  stakeholders.  In  part,  we  were  seeking  to 
improve  our  understanding  of  the  customer  side  of  the  logic  model,  including  the  ways 
in  which  those  relationships  might  lead  to  intended  outcomes  and  support  successful 
UOCAVA  voting. 


Our  Approach  to  Stakeholder  Outreach 

The  logic  model  provided  us  with  a  point  of  departure  for  identifying  particular  stake¬ 
holders  of  interest,  consisting  of  institutions  and  individuals.  In  addition,  we  asked 
FVAP  staff,  including  leadership,  for  their  input  on  important  contacts.  We  reached 
out  to — but  did  not  always  make  contact  with — a  considerable  share  of  those  who 
regularly  interfaced  with  FVAP  as  partners  and  intermediate  customers.1 

The  stakeholders  included  members  of  five  distinct  stakeholder  groups  and  one 
catch-all  category  of  advocates:2 

•  congressional  staff 

•  election  officials  and  representative  NGOs 

•  overseas-citizen  NGOs 

•  DoD  entities,  e.g.,  IG,  general  counsel,  DFIRA,  and  representatives  of  the  uni¬ 
formed  services  (SVAOs)3 

•  academics  and  technologists  with  an  interest  in  UOCAVA  voting 

•  various  election  advocates. 


1  We  reached  out  to  local  election  officials  through  a  representative  NGO. 

2  We  held  our  conversations  over  a  period  spanning  about  three  months,  beginning  in  July  2013. 

3  We  conducted  site  visits  and  reached  out  to  other  DoD  entities  in  phase  2  of  the  project. 
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The  stakeholders  represented  wide-ranging  interests  and,  in  many  instances,  have 
had  ongoing  contact  with  FVAP  over  a  period  of  many  years.  Not  surprisingly,  inter¬ 
ests  differed — and  sometimes  conflicted — both  across  and  within  stakeholder  groups, 
such  as  those  consisting  of  congressional  staff  and  election  officials. 

Illustrative  of  that  diversity,  we  did  the  following: 

•  engaged  with  congressional  staff  at  relevant  committees — namely,  the  U.S.  Senate 
Committee  on  Armed  Services,  U.S.  House  of  Representatives  Committee  on 
Armed  Services,  U.S.  Senate  Committee  on  Rules  and  Administration,  and  U.S. 
House  of  Representatives  Committee  on  Administration — and  other  staff  who 
routinely  engage  on  UOCAVA  voting  issues 

•  reached  out  to  election  officials,  consisting  of  those  in  attendance  at  the  National 
Association  of  State  Election  Directors  (NASED)  and  National  Association  of 
Secretaries  of  State  (NASS)  summer  conferences  held  in  Anchorage,  Alaska,  and 
to  NGOs  (namely,  NASED,  NASS,  and  the  Election  Center)  that  represent  elec¬ 
tion  officials 

•  reached  out  to  overseas- citizen  NGOs,  including  the  Overseas  Vote  Foundation, 
Union  of  Overseas  Voters,  Association  of  Americans  Resident  Overseas,  and  the 
Federation  of  American  Women’s  Clubs  Overseas. 

In  each  case,  we  sought  to  learn  more  about  the  stakeholder’s  view  of  FVAP,  but 
we  did  not  undertake  formal  interviews;  rather,  we  engaged  discussants  in  unstruc¬ 
tured,  open-ended  conversations.  To  introduce  ourselves  and  the  project,  we  expressed 
an  interest  in  learning  more  about  how  stakeholders  see  FVAP,  how  they  see  their  rela¬ 
tionships  with  FVAP,  what  they  need  from  FVAP,  and  what  they  get  and  do  not  get 
from  FVAP.  We  wanted  to  learn  more  about  the  successes  and  failures  of  the  relation¬ 
ships  and  the  ways  in  which  they  might  lead  to  intended  outcomes  and  support  suc¬ 
cessful  UOCAVA  voting,  but  we  left  it  to  each  discussant  to  determine  the  direction 
of  the  conversation.  In  so  doing,  we  were  able  to  benefit  from  the  spontaneity  of  each 
encounter,  which  reflected  the  priorities  of  each  stakeholder. 


Insights  from  Stakeholder  Outreach 

Perhaps  most  striking  among  the  insights  that  we  drew  from  these  conversations  was 
the  extent  to  which  the  apparent  divergence  of  interest  among  stakeholders  yielded  a 
broad  convergence  around  common  themes. 

In  Table  3.1,  we  present  a  summary  of  those  convergences  along  with  comments, 
caveats,  and  examples.  In  the  spirit  of  convergence,  the  table  includes  only  those  themes 
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Table  3.1 

Broad  Convergences  Across  Stakeholder  Groups 


Convergence 


Comment,  Caveat,  or  Example 


Stick  to  the  knitting 
Recognize  voters  as  customers 


Be  a  trusted  source 

Be  a  trusted  partner 

Communicate  more,  more  regularly, 
and  better 


Improve  transparency  of  processes, 
practices,  and  organization 

Do  no  harm 

Understand  the  operating  environment 
Manage  institutional  constraints 


Examples:  "stick  to  the  core,"  "do  one  thing  well" 

Is  FVAP's  role  to  ensure  that  voters  can  register  or  do 
register? 

Is  FVAP's  role  to  inform  voters  or  to  motivate  voting? 
Do  some  UOCAVA  voters  count  more  than  others? 

FVAP  cannot  be  or  appear  to  be  political. 

FVAP  must  set  a  high  bar  for  itself  and  for  others. 


Reach  out,  listen  to  others,  and  respond. 

Target  and  tailor  the  message. 

Send  the  right  messengers. 

Strike  an  appropriate  balance  between  modes  of  assis¬ 
tance  (IT,  paper,  and  humans). 

Act  as  openly  as  possible. 

Specify  points  of  contact. 

Post  data  and  research  results  promptly  and  visibly. 

Post  criteria  for  decisionmaking. 

If  possible,  make  things  better. 

Act  deliberately,  thoughtfully,  and  efficiently  with  FVAP's 
own  and  others'  resources. 

Voting  is  part  of  a  system,  but  where  is  FVAP  in  that 
system  (e.g.,  is  it  in  operations,  policy,  or  both)? 

FVAP  cannot  change  constraints  but  must  navigate  them 
effectively  or  more  effectively. 


SOURCE:  RAND  staff  analysis  based  on  conversations  with  FVAP  stakeholders. 


that  cut  across  stakeholder  groups.4  Although  we  emphasize  that  we  are  reporting  on 
the  stakeholders’  views  and  not  our  own,  we  also  acknowledge  that  we  have  synthe¬ 
sized  and  bundled  their  views  into  categories,  sometimes  with  new  or  different  vocabu¬ 
lary.  For  example,  we  use  the  phrase  “stick  to  the  knitting”5  (meaning  “stay  on  point,” 
“devote  yourself  to  your  responsibilities,”  and  “do  what  you  know  how  to  do  well”) 
to  encapsulate  the  groups’  observations  and  concerns  about  FVAP’s  focus — or  lack 
thereof — and  overall  approach  to  service  provision. 


4  Except  insomuch  as  the  “comments,  caveats,  and  examples”  reference  specific  points  raised  by  particular 
stakeholders  or  groups  of  stakeholders,  the  entries  in  Table  3.1  do  not  speak  to  particular  issues  that  were  relevant 
to  only  one  or  two  groups.  For  example,  we  do  not  include  the  SVAOs’  comments  on  VAO  training  or  IVA  offices 
in  Table  3.1  but  took  them  into  consideration  in  our  analysis  of  FVAP’s  work  with  the  military  in  Chapter  Six. 

5  Peters  and  Waterman,  1982,  have  been  credited  with  popularizing  the  term  in  their  management  text,  In 
Search  of  Excellence.  They  defined  the  term  as  “staying  within  the  business  you  know  best,”  but  others  impart 
broader  meaning.  Although  the  book  has  been  subject  to  criticism,  the  term  serves  our  purposes  in  this  chapter. 


32  The  Federal  Voting  Assistance  Program  and  the  Road  Ahead 


Notwithstanding  the  convergences  around  the  themes  represented  in  Table  3.1, 
discussants  across  and  within  stakeholder  groups  sometimes  offered  very  different 
insight  to  the  underlying  concerns  and  their  remedies.  Here,  we  highlight  some  of  the 
subtleties  beneath  the  table  entries,  but  we  save  consideration  of  “knitting”  for  the  close 
of  this  chapter. 

Recognize  Voters  as  Customers 

Discussants  in  nearly  all  stakeholder  groups  addressed  the  centrality  of  voters  as  cus¬ 
tomers,  but  they  differed  in  their  views  about  how  FVAP  should  be  serving  them: 

•  Some  discussants  indicated  that  FVAP  should  be  ensuring  that  voters  can  register 
to  vote;  others  suggested  that  it  should  be  ensuring  that  they  do  register. 

•  Differences  also  emerged  in  their  views  of  FVAP’s  position  on  awareness  and 
motivation;  that  is,  should  the  agency  be  raising  individuals’  awareness  of  the 
right  and  resources  to  vote,  or  should  it  be  motivating  them  to  vote? 

•  Discussants,  both  within  and  across  communities,  also  voiced  divergent  opinions 
about  the  relative  importance  of  uniformed-service  and  overseas  voters  and  the 
balance  of  effort  that  should  be  allocated  to  them. 

The  issue  of  trust  came  up  explicitly  and  implicitly  across  all  groups,  with  obser¬ 
vations  falling  into  two  related  categories:  one  pertained  to  trust  in  FVAP  as  a  source 
of  information  and  the  other  to  trust  in  FVAP  as  a  voting  assistance  partner.  But  dis¬ 
cussants  differed  in  their  opinions  of  FVAP’s  trustworthiness.  Whereas  some  lauded 
FVAP  as  a  trusted  source  to  which  they  would  and  do  readily  steer  voters  (e.g.,  to  use 
the  Voting  Assistance  Guide  or  online  assistants),  others  framed  trust  as  an  aspiration. 

Be  a  Trusted  Source  and  Partner 

In  conceptualizing  a  trusted  source,  discussants  focused  on  the  importance  of  avoid¬ 
ing  the  appearance  of  partisanship,  politicization,  or  salesmanship  and  on  the  quality 
and  reliability  of  services  and  products.  If  FVAP  appeared  to  be  pushing  something, 
the  agency  did  not  seem  to  fare  well  in  a  stakeholder  group’s  characterization.  Among 
those  discussants  focusing  on  quality  and  reliability,  some  suggested  that  FVAP  should 
model  the  ideal  or  be  best  in  class  for  the  voting  community.  More  concretely,  one 
discussant  suggested  that  FVAP  develop  training  standards  for  use  within  the  U.S. 
government  and  across  NGOs. 

The  need  for  a  trusted  partner  was  evident,  if  not  always  articulated,  in  discus¬ 
sants’  observations  on  communication  and  transparency. 

Discussants,  almost  uniformly,  spoke  to  a  need  for  FVAP  to  reach  out  to  stake¬ 
holders  more  often  and  more  regularly,  listen  to  what  they  have  to  say,  and  respond. 
They  noted  repeatedly  that  communication  needed  to  be  ongoing  and  routinized 
and  that  listening — by  itself,  absent  processing  and  responding,  whether  positively  or 
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negatively — was  not  enough.  In  effect,  F  VAP  should  treat  its  partners  as  active  collabo¬ 
rators,  not  as  passive  operatives  or,  in  some  instances,  competitors.  The  tenor  of  these 
conversations  suggested  that  the  creation  of  open,  two-way  channels  might  require 
time  and  effort  but  that  FVAP  could  benefit  from  them.  FVAP  might  garner  increased 
support  from  stakeholders,  eliminate  misperceptions  about  its  intentions  and  actions, 
and,  perhaps,  decrease  the  incidence  of  rapid  responses  to  perceived  crises  or  firefight¬ 
ing.  FVAP  might  also  find  opportunities  to  leverage  stakeholders’  resources.  Overseas 
NGOs,  for  example,  might  be  able  to  offer  access  to  their  regional  and  global  customer 
bases — a  reach  that  FVAP  could  not  reproduce  cost-effectively,  if  at  all. 

Discussants  across  groups  also  signaled  the  importance  of  targeting6  and  tailor¬ 
ing  messages  to  the  needs  of  particular  stakeholders  and,  in  the  case  of  direct,  person- 
to-person  engagement,  sending  appropriate  messengers  (for  example,  by  ensuring  that 
people  with  appropriate  expertise,  not  just  bureaucratic  standing,  attend  meetings).7 

Although  less  clearly  linked  to  trust,  the  issue  of  the  balance  between  the  use  of 
IT,  paper,  and  human  interaction  in  providing  assistance  also  emerged.  We  can  recall 
no  discussant  saying  that  one  method  alone  would  be  adequate.  Most  indicated  the 
desirability  of  diverse  approaches  and  a  mixed  portfolio,  and  some  suggested  the  need 
for  a  deeper  consideration  of  the  appropriate  balance  among  them.  Some  stakeholders 
also  noted  that  the  FVAP  website  had  come  a  long  way  in  providing  service  to  voters 
(e.g.,  praising  the  utility  of  online  tools)  but  suggested  that  it  still  had  some  distance  to 
travel  in  serving  others,  in  terms  of  both  navigability  and  content. 

The  issue  of  transparency — with  respect  to  processes,  practices,  and  organiza¬ 
tional  structure — also  featured  prominently  in  our  conversations,  sometimes  closely 
related  to  the  issues  of  listening  and  responding  in  communication: 

•  Discussants  across  all  groups  emphasized  the  importance  of  FVAP  acting  as 
openly  as  possible.  To  illustrate,  they  compared  two  recent  efforts  to  revise  the 
FPCA  and  FWAB  forms.  They  depicted  the  first  effort  as  one  in  which  FVAP 
staff  did  not  solicit  or  respond  uniformly  to  external  input,  resulting  in  consid¬ 
erable  dissatisfaction,  if  not  outright  hostility,  among  some  stakeholders.  They 
depicted  the  second  effort,  run  through  OMB,  as  open,  constructive,  and  yield¬ 
ing  some  buy-in  from  stakeholders.8  Discussants  held  up  the  latter  process  as  a 
positive  example  of  transparency — and  of  listening  and  responding.9 


6  In  this  regard,  some  suggested  that  too  much  information  can  be  as  problematic  as  too  little. 

7  Along  similar  lines,  a  representative  of  an  overseas  citizens’  group  asked  that  those  conducting  offshore  work¬ 
shops  and  trainings  be  well-acquainted  with  on-the-ground  conditions. 

8  We  discuss  this  process  in  greater  detail  in  Chapter  Seven. 

9  They  acknowledged  that  not  everyone  could  or  would  get  what  they  wanted  in  the  new  FPCA  and  FWAB,  but 
they  would  know  they  were  heard  and,  if  their  views  were  not  taken  on  board,  they  would  know  why. 
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•  Discussants  across  most,  if  not  all,  groups  suggested  that  data  and  research  results 
should  be  posted  promptly  and  visibly  and  noted  that  failing  to  do  so  can  and 
probably  has  created  the  impression  of  purposeful  obfuscation. 

•  Similarly,  they  suggested  that  FVAP  post  criteria  for  making  decisions,  e.g.,  in 
altering  policy  directions  and  defining  terms,  such  as  installation. 

•  Discussants  also  spoke  to  the  importance  of  organizational  transparency,  citing 
a  need  to  know  who  does  what  in  the  agency  and  how  to  reach  them.  Some,  for 
example,  recounted  challenges  reaching  the  right  person  in  the  right  moment  and 
asked  for  a  readily  accessible  list  of  points  of  contact  and  contact  information. 

Do  No  Harm 

The  point  that  FVAP  should  do  no  harm  surfaced  in  several  conversations.  Of  primary 
concern  were  the  integrity  of  the  voting  system10  and  the  use  of  resources  throughout 
the  system.  In  the  case  of  resourcing,  the  discussants  appeared  to  be  concerned  with 
how  FVAP’s  actions  were  affecting  the  use  of  its  and  others’  resources  and  to  be  asking 
FVAP  to  act  more  deliberately,  thoughtfully,  and  efficiently  with  those  resources.  Dis¬ 
cussants  spoke  to  incidents  of  firefighting  and  to  ordinary  features  of  their  relation¬ 
ships  with  FVAP.  Regarding  firefighting,  discussants  across  groups — and  invariant  to 
the  specifics  of  any  particular  observation — described  a  pattern  of  rapid  responses  to 
perceived  crises,  leading  FVAP  to  change  course  and  place  new  and  mostly  additional 
demands  on  the  voting  community  absent  a  clear  benefit.  Regarding  the  ordinary, 
including  requests  from  FVAP  for  election  data,  discussants  also  questioned  whether 
the  benefits  justified  the  costs. 

Understand  the  Operating  Environment  and  Manage  Institutional  Constraints 

Some  discussants  observed  or  implied  that  FVAP  would  benefit  from  thinking  about 
its  operating  environment  and  the  voting  community  as  a  system — one  in  which  func¬ 
tioning  depends  partly  on  policy  decisions  made  by  FVAP  and  actions  taken  by  FVAP. 
In  so  doing,  the  agency  might  be  better  able  to  anticipate  the  repercussions  of  policy 
decisions  and  actions  for  its  stakeholders.  A  few  discussants  also  raised  questions  about 
FVAP’s  role  in  the  system — e.g.,  whether  it  is  to  make  policy,  to  implement  policy,  or 
both — and  its  necessity. 


10  Here,  we  are  referring  not  just  to  the  technology  components  of  the  system  but  to  the  relational  components, 
e.g.,  among  those  who  routinely  provide  hands-on  assistance  to  voters. 
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Related  to  the  issue  of  environment,  discussants  acknowledged  that  FVAP  must 
contend  with  a  nontrivial  set  of  institutional  constraints,  some  of  which  might  make  it 
difficult  for  the  agency  to  accomplish  its  mission,  however  defined: 

•  the  absence  of  leadership  and  leadership  continuity  within  FVAP 

•  the  agency’s  position  in  DoD  and  the  implications  of  that  position  for  the  agen¬ 
cy’s  authority  and  the  accountability  of  its  partners 

•  the  relative  importance — or  lack  thereof — of  voting  issues  among  all  DoD  issues 

•  the  cyclicality  of  DoD’s  interest  in  voting 

•  a  potential  lack  of  concern  for  overseas  civilian  voters  within  DoD,  if  not  FVAP 

•  DoD’s  workplace  culture  with  regard  to  transparency 

•  the  autonomy  of  the  states. 

Discussants  suggested  that  the  act  of  voting  is  not  at  the  center  of  DoD’s  mission 
nor  is  it  a  front-burner  issue  most  of  the  time,  which  could  both  impede  FVAP’s  efforts 
and  make  it  possible  for  the  agency  to  drift.  They  also  noted  that  past  FVAP  leadership 
had  either  tended  to  pull  the  agency  in  different  directions  or,  sometimes,  lacked  clear 
direction.  Some  also  suggested  that  engagement  with  stakeholders  had  been  driven  too 
much  by  the  agendas  of  people  within  the  agency  and  not  enough  by  the  mission  of 
the  agency. 

None  of  the  discussants  suggested  giving  FVAP  leeway  because  of  the  constraints; 
rather,  they  indicated  that  the  agency  should  learn  to  navigate  the  constraints  more 
effectively. 

The  second-most  striking  finding  from  our  outreach  was  the  lack  of  interest  or 
awareness  among  some  stakeholders.  Whereas  many  of  the  stakeholders  that  we  con¬ 
tacted  spoke  about  FVAP  with  passion,  more  than  a  handful  were  not  interested  in  dis¬ 
cussing  FVAP11  or  lacked  awareness  of  the  agency;  the  activities  it  undertakes;  what  it 
produces;  and  how  the  agency,  activities,  and  outputs  relate  to  them.  Given  the  extent 
of  our  efforts  to  target  institutions  and  people  who  seemed  likely  to  have  an  interest  in 
and  awareness  of  FVAP,  this  might  have  been  surprising;  however,  noting  that  “FVAP 
and  the  Road  Ahead”  was  not  the  first  strategically  oriented  study  that  FVAP  had  com¬ 
missioned  in  recent  history,  some  amount  of  outreach  fatigue  could  explain  the  lack  of 
interest,  if  not  the  lack  of  awareness. 


Concluding  Remarks  on  Stakeholder  Outreach 

We  conducted  stakeholder  outreach,  in  part,  because  we  needed  to  know  more  about 
the  use  and  ultimate  effects  of  the  agency’s  outputs  and  the  strength  of  its  relationships 


11 


For  example,  they  did  not  reply  to  our  emails. 
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with  partners  and  intermediate  customers.  Our  open-ended  conversations  yielded  a 
multifaceted  picture  of  stakeholders’  perceptions  of  FVAP’s  mission,  priorities,  and 
activities  and  its  role  in  the  voting  community.  With  those  perceptions  in  hand,  we 
were  able  to  begin  testing  and  validating  our  understanding  of  the  agency  and  explor¬ 
ing  the  effectiveness  and  value  of  FVAP’s  work  with  stakeholders.  As  we  address  in 
greater  depth  in  Chapter  Five,  we  uncovered  substantive  differences  between  FVAP’s 
perception  of  itself,  to  the  extent  that  those  perceptions  were  embedded  in  the  bench¬ 
mark  logic  model,  and  the  stakeholders’  perceptions  of  F VAP. 

We  close  this  chapter  with  a  discussion  of  the  phrase  “stick  to  the  knitting,” 
because  the  underlying  issues  were  central  to  so  many  of  our  conversations  and  because 
it  provides  a  natural  bridge  to — and  a  reason  for — the  assessment  of  requirements  that 
follows  in  the  next  chapter.  Discussants  suggested  that  FVAP  “stick  to  its  core,”  “do 
one  thing  well,”  or  “do  what  it  does  best”  and,  at  times,  described  FVAP  as  scattered, 
spread  too  thin,  and  overextended  in  too  many  arenas.  They  lacked  agreement  on  what 
that  knitting  should  look  like  but  leaned  toward  assigning  prominence  to  hands-on 
or  direct  assistance  and  looking  to  the  law  for  guidance.  As  evidence,  we  note  that 
stakeholders  seemed  to  speak  most  positively  of  FVAP  when  they  recounted  stories  of 
troubleshooting  to  meet  immediate  needs.12 

The  next  chapter  demarcates  a  set  of  core  activities,  primarily  from  a  legal  per¬ 
spective.  However,  even  if  the  law  provides  mandates  or  suggests  certain  activities,  e.g., 
with  “shall”  or  “may”  language,  it  was  not  written  as  a  mission  statement,  an  organi¬ 
zational  chart,  or  a  manual  and  cannot  answer  all  questions  about  FVAP’s  operations. 


12  For  example,  an  election  official  described  a  time-sensitive  circumstance  in  which  an  FVAP  staff  member  went 
to  great  lengths  to  help  the  state  understand  and  meet  its  UOCAVA  responsibilities. 


CHAPTER  FOUR 


Federal  Voting  Assistance  Program  Activities  in  Relation  to 
Requirements 


In  this  chapter,  we  discuss  the  findings  from  our  assessment  of  the  relationships  among 
FVAP  activities,1  U.S.  law  on  UOCAVA  voting,  and  DoD  policy  as  of  November 
2013.  The  relevant  laws  are  codified  primarily  in  U.S.  Code  (U.S.C.),  Title  10,  §  1566 
and  §  1566a;  42  U.S.C.  §§  1973ff  et  seq.;  42  U.S.C.  §§  1973gg  et  seq.;  and  42  U.S.C. 
§  15344.2  The  discussion  of  policy  draws  from  DoDI  1000.04,  2012,  the  subject  of 
which  is  Federal  Voting  Assistance  Program  (FVAP). 

This  purpose  of  this  assessment  was  to  identify  the  minimum  set  of  activities  that 
FVAP  must  undertake  to  meet  specific,  direct  requirements,  to  which  we  refer  as  the 
core,  and  to  improve  our  understanding  of  how  FVAP’s  activities  relate  to  that  core. 
We  were  looking  both  for  gaps  in  coverage  and  for  potential  opportunities  for  redirec¬ 
tion.  A  noncore  activity  might  contribute  importantly  to  the  assistance  of  UOCAVA 
voters,  either  directly  or  indirectly,  by  supporting  core  activities,  meeting  more-general 
requirements,  or  filling  other  institutional  needs.  Nevertheless,  FVAP  might  have  lati¬ 
tude  to  consider  alternative  courses  of  action,  depending  on  institutional  priorities  and 
resource  availability. 

Because  DoD,  including  FVAP,  can  elicit  changes  in  policy  through  revisions  of 
DoDDs  and  DoDIs  more  readily  than  it  can  elicit  changes  in  law,  we  treated  policy 
statements  as  less  constraining  than  legal  mandates  and  looked  to  the  law  first  in  our 
development  of  the  must-do  list;  that  is,  we  singled  out  a  set  of  specific,  direct  legal 
requirements,  which  we  used  as  the  foundation  for  identifying  the  core.  FVAP,  under 
current  practice,  holds  the  pen  in  the  process  of  revising  and  reissuing  DoDI  1000.04, 


1  We  use  the  present  tense  to  describe  FVAP’s  activities  in  this  chapter,  but  note  that  our  period  of  analysis 
ended  in  November  2013  and  that,  since  that  time,  FVAP  has  made  changes  to  specific  practices  in  a  variety  of 
areas.  When  we  are  aware  of  such  a  change,  we  note  the  change  in  our  discussion. 

2  Following  the  completion  of  the  analysis  for  this  report,  the  U.S.C.  designations  for  voting-related  provisions, 
including  those  in  the  totality  of  §§  1973ff  and  §§  1973gg,  were  recast  under  their  own  title,  specific  to  voting, 
i.e.,  Title  52,  Voting  and  Elections.  For  the  new  code  and  a  crosswalk,  see,  respectively,  U.S.C.  Title  52  and  Office 
of  the  Law  Revision  Counsel,  undated. 
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which  is  the  primary  (DoD)  policy  document  concerning  UOCAVA  voting.3  It  cannot, 
however,  dictate  the  content  of  that  document  because  it  must,  like  all  agencies,  run 
the  process  in  accordance  with  standard  practices  and  procedures  that  involve  inter¬ 
agency  and  external  feedback,  input,  and  approval.  Moreover,  once  the  instruction  is 
issued,  FVAP,  like  all  other  participants  in  the  system,  must  uphold  the  terms  of  the 
instruction. 

We  approached  the  analysis  from  two  directions.  First,  we  applied  the  lens  of  the 
law,  asking,  “What  does  the  law  require  of  FVAP,  specifically  and  directly,  and  what 
is  FVAP  doing  to  meet  those  requirements?”  Second,  we  looked  through  the  lens  of 
FVAP  activities  and  asked,  “What  activities  does  FVAP  undertake  to  serve  UOCAVA 
voters,  and  in  what  ways  does  the  law  require  them?”  To  address  the  first  question,  we 
created  a  checklist  of  direct,  specific  legal  requirements  and  then  considered  whether 
FVAP  activities — as  depicted  in  the  logic  model,  apparent  on  the  FVAP  website,  or 
discussed  with  FVAP  staff  and  stakeholders — addressed  them.  Then,  we  created  a  list 
of  activities,  which  we  mapped  to  the  requirements. 

In  comparing  the  law  to  FVAP  activities  and  FVAP  activities  to  the  law,  our  anal¬ 
ysis  addressed  what  FVAP  was  doing  and  not  whether  it  was  doing  it  well. 

Figure  4.1  depicts  a  generic,  untested  relationship  between  an  unnamed  agency’s 
activities,  core  legal  requirements,  and  policy  and  frames  these  questions  as  a  Venn  dia- 

Figure  4.1 

Generic  Depiction  of  Activities,  Law,  and  Policy 


RAND  RR882-4. 7 


^  FVAP  acts  on  behalf  of  USD(P&R),  who  issues  the  guidance.  See  DoDD  5124.02,  2008,  for  an  explanation 
of  the  delegation  of  responsibilities  to  USD(P&R). 
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gram.  The  goal  of  the  analysis  in  this  chapter  was  to  define  the  content  of  the  law  oval, 
i.e.,  the  core  requirements,  and  to  determine  the  relative  positions  of  the  ovals.  Are 
they  partially  overlapping,  roughly  equivalent,  or  grossly  divergent?  The  policy  oval  is 
dashed  and  colorless  both  because  we  view  policy  as  less  binding  than  law  and  because 
it  played  a  lesser  role  in  our  analysis. 


Through  the  Lens  of  the  Law 

In  this  section,  we  discuss  our  approach  to  identifying  the  set  of  core  legal  require¬ 
ments  and  our  findings  on  FVAP’s  satisfaction  of  those  requirements. 

To  identify  those  requirements,  we  drew  from  the  provisions  in  10  U.S.C.  §  1566 
and  §  1566a  and  in  42  U.S.C.  §§  1973ff  et  seq.  and  42  U.S.C.  §  15344  that  could 
assign  responsibilities  to  FVAP  as  the  “federal  voting  assistance  program”  or  the  del¬ 
egate  of  the  “presidential  designee.”  In  general,  we  did  not  consider  legal  provisions  that 
named  the  “Secretary  of  Defense,”  “Secretary,”  or  “Department  of  Defense”  without 
specific  reference  to  the  presidential  designee  or  federal  voting  assistance  program.4 
ITowever,  as  an  exception,  we  included  the  Electronic  Voting  Demonstration  Project. 
The  provision  that  established  the  project  did  not  assign  responsibility  to  either  the 
presidential  designee  or  a  federal  voting  assistance  program,  but  the  project  has  been 
associated  closely  with  FVAP  through  appropriations,  policy,  and  practice.5  We  also 
included  one  broadly  worded  provision  found  in  42  U.S.C.  §§  1973gg  et  seq.  that  calls 
on  federal,  state,  and  local  governments  to  promote  the  exercise  of  the  right  to  vote.6 

It  is  important  to  note  that  even  the  provisions  that  specifically  assign  responsibil¬ 
ity  to  the  presidential  designee  or  a  federal  voting  assistance  program  do  not  necessarily 
create  a  requirement  for  FVAP.  For  example,  the  Secretary  of  Defense  could  choose  to 
delegate  responsibility  elsewhere  and  appears  to  have  done  so  in  the  cases  of  some  pro¬ 
visions  concerning  ballot  collection  and  delivery.  In  those  cases,  responsibility,  as  set 
out  in  DoDI  1000.04,  2012,  rests  with  MPSA.  Moreover,  it  is  sometimes  the  case  that 
the  phras  e.  federal  voting  assistance  program  is  used  to  encompass  all  federal  voting  assis¬ 
tance,  whether  or  not  it  is  tied  to  FVAP,  the  agency.  Nevertheless,  in  the  discussions 
that  follow,  we  refer  to  the  law  as  implying  requirements  for  FVAP — and  treat  them 


4  We  do  not,  for  example,  reference  certain  provisions  that  address  regulatory  or  decisionmaking  authority  (see 
10  U.S.C.  §  1566[a]  and  §  1566a[a],  [c],  [e],  and  [f])  or  delivery  of  mail  from  overseas  (see  10  U.S.C.  §  1566[g]). 
Similarly,  we  do  not  include  provisions  found  in  42  U.S.C.  §  1973gg-5,  Voter  Registration  Agencies.  However, 
we  note  that  DoDI  1000.04,  2012,  provided  a  role  for  FVAP  in  the  implementation  of  42  U.S.C.  §  1973gg-5. 

5  Subsequent  to  this  analysis,  Section  593  of  the  National  Defense  Authorization  Act  for  Fiscal  Year  2015,  titled 
“Repeal  of  Electronic  Voting  Demonstration  Project,”  eliminated  the  requirement. 

6  One  could  argue  that  this  provision  applies  to  the  federal  government  and,  hence,  would  apply  to  FVAP 
because  it  is  a  part  of  the  federal  government. 
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as  core  requirements — when  it  specifies  the  presidential  designee  or  the  federal  voting 
assistance  program,  unless  we  have  evidence  that  it  means  otherwise. 

Table  4.1  summarizes  the  statutory  requirements.  In  the  third  column  of  the 
table,  we  address  corresponding  activities  and  cite  evidence,  sometimes  in  the  form  of  a 
web  page  that  was  active  at  the  time  of  our  analysis;  however,  we  note  that,  in  the  time 
that  has  elapsed  since  the  completion  of  the  analysis,  F  VAP  has  undertaken  a  substan¬ 
tial  redesign  of  its  website,  and  many  or  most  of  those  web  addresses  (uniform  resource 
locators,  or  URLs)  are  no  longer  active. 

For  the  text  of  the  relevant  provisions,  see  Appendix  B. 

The  law,  at  least  implicitly,  also  mandates  that  FVAP  form  relationships  with  a 
variety  of  entities,  including  the  states  and  localities;  EAC;  NIST;  USPS;  GSA;  DOJ; 
Congress;  other  U.S.  government  departments,  agencies,  and  entities;  and  other  DoD 
entities,  including  the  uniformed  services,  MPSA,  and  the  IG.  These  institutions  are 
represented  in  the  benchmark  logic  model  as  stakeholders. 


Through  the  Lens  of  Federal  Voting  Assistance  Program  Activities 

In  this  section,  we  discuss  our  attempt  to  map  FVAP  activities  to  the  particular  pro¬ 
visions  of  law,  listed  in  Table  4.1.  Taking  this  perspective  allowed  us  to  account  for 
the  possibility  that  FVAP  activities  could  be  addressing  specific,  direct  requirements 
of  law;  supporting  such  requirements  or  addressing  other  less  specific  requirements; 
or  addressing  other  institutional  (FVAP,  DoD,  or  broader  U.S.  government)  needs, 

Table  4.1 

Statutory  Requirements  Pertaining  to  the  Federal  Voting  Assistance  Program 


Corresponding  Activity  or  Location  of 
Title  and  Section  or  Information  About  the  Activity,  as  of 

Subsection  Summary  of  Requirement  November  2013 


FVAP  and  presidential-designee  requirements 
State-oriented  provisions 


42  U.S.C. 

§  1 973ff  (b)(1) 


Consult  state  and  local 
election  officials  in  carrying 
out  UOCAVA,3  and  ensure 
their  awareness  of  the 
requirements  of  UOCAVA 


42  U.S.C.  §  1973ff-  Approve  or  deny  state 
1(g)  waiver  requests 


42  U.S.C.  §  1973ff- 
2(e) 


Approve  or  deny  state 
absentee  ballot  in  place  of 
FWAB 


Conference  participation,  email  alerts, 
and  other  outreach;  online  training  and 
guidance1-* 


See  FVAP  website — in  particular,  the 
link  to  "Move  Act  Waivers"  (www.fvap. 
gov/reference/laws/waivers),  found 
under  "Reference/Reports"  and  "Laws" 

No  evidence  of  recent  need,  but  FVAP 
appears  to  have  means  to  address 
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Table  4.1 — Continued 


Corresponding  Activity  or  Location  of 
Title  and  Section  or  Information  About  the  Activity,  as  of 

Subsection  Summary  of  Requirement  November  2013 


State-  and  voter-oriented  provisions 

42  U.S.C.  Compile  and  distribute 

§  1 973ff (b)(5)  descriptive  material  on 

state  procedures  and, 
to  the  extent  possible, 
election  facts 


42  U.S.C.  §  1973ff- 
1(e)(4) 


Maintain  and  make 
publicly  available  online 
repository  of  state  contact 
information 


Voter-oriented  provisions 


42  U.S.C. 

Prescribe  forms  (FPCA, 

§  1973ff (b)(2),  (4), 

FWAB,  including 

and  (7);  42  U.S.C. 

instructions),  envelope 

§  1973ff-2 

design,  and  standard  oath 

42  U.S.C. 

Carry  out  sections  §  1973ff- 

§  1 973ff (b)(3),  (8), 

2  (for  FWAB);  §  1973ff-2a 

and  (10) 

(for  ballot  collection  and 
delivery);  and  §  1973ff- 
2b  (for  Federal  Voting 
Assistance  Program 
improvements) 

42  U.S.C. 

Ensure  privacy  and 

§  1973ff  (b)(9) 

independence  of 
uniformed-service  voters  at 
locations  under  designee's 
jurisdiction,  and  protect 
privacy  of  ballots  under 
designee's  control 

42  U.S.C.  §  1973ff- 

Develop  procedures  to 

2(a)(2)(A) 

promote  and  expand  use  of 
FWAB 

42  U.S.C.  §  1973ff- 

Use  technology  to 

2(a)(2)(B) 

implement  a  system  to 
obtain  list  of  candidates 
and  submit  printed  ballots 

Online  and  print  (2012-2013)  versions 
of  the  Voting  Assistance  Guide  (www. 
fvap.gov/vao/guide;  FVAP,  201 3f) 

FVAP  website — for  example,  the  link 
to  "Select  Your  State"  (www.fvap. 
gov/map),  found  under  "State  Voting 
Information" 

FVAP  website — in  particular,  the  link 
to  "Find  Out  Where  to  Send  Election 
Materials"  (www.fvap.gov/shortcuts/ 
wheresendfpca),  found  under  "Where 
to  Send  Election  Materials" 

Online  version  of  the  Voting  Assistance 
Guide  (www.fvap.gov/vao/guide) 


FVAP  website — in  particular,  the  link  to 
"Forms  Available  to  Download"  (www. 
fvap.gov/reference/forms),  found  under 
"Reference/Reports" 

See  respective  entries  for  relevant  sec¬ 
tions  (i.e.,  §  1973ff-2,  §  1973ff-2a,  and 
§  1973ff-2b)  elsewhere  in  this  table 


It  is  our  understanding  that  FVAP  takes 
steps  to  ensure  privacy  and  provides 
notices  in  relevant  forms  (e.g.,  FWAB 
and  FPCA)  when  it  cannot 


Absentee-voter  weeks,  email  alerts, 
and  other  outreach;  revising  and 
updating  forms  and  providing  tools13 

FVAP  website — for  example,  the  link 
to  "Get  Started"  (www.fvap.gov/reg- 
istration/reg-ballot-request-pa),  found 
under  "State  Information,"  "PA," 
then  "Register  to  Vote  and/or  Request 
Ballot" 
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Table  4.1 — Continued 


Corresponding  Activity  or  Location  of 
Title  and  Section  or  Information  About  the  Activity,  as  of 

Subsection  Summary  of  Requirement  November  2013 


42  U.S.C.  §  1973ff- 
2a(a),  (b) 

Establish  procedures  for 
collecting  and  delivering 
marked  absentee  ballots  of 
absent  overseas  uniformed- 
service  voters,  implement 
procedures  to  facilitate 
delivery,  and  provide 
expedited  delivery,  in 
cooperation  with  USPS 

42  U.S.C.  §  1973ff- 
2a(b)(3) 

May  establish  alternative 
deadlines  for  certain 
locations 

42  U.S.C.  §  1973ff- 
2a(c) 

Conduct  outreach  to  inform 
people  of  procedures,  as 
covered  under  42  U.S.C. 

§  1973ff-2a(a),  (b) 

42  U.S.C.  §  1973ff- 
2b 

Develop  online  portals 
(to  access  procedural 
information)  and 
notification  program 

10  U.S.C. 

§  1566a(d) 

Conduct  outreach  to 
inform  absent  uniformed- 
service  voters  of  assistance 
(§  1566a) 

Delegated  in  part  or  whole  to  MPSA 


Delegated  in  part  or  whole  to  MPSA 


Shared  with  MPSA 

Broadcasts  through  email  blasts  and 

other  dissemination  channels 


Work  ongoing  (FVAP,  2011a) 
Broadcasts  through  email  blasts  and 
other  dissemination  channels 


Shared  with  uniformed  services 
Broadcasts  through  email  blasts  and 
other  dissemination  channels 


Data,  reporting,  and  other  research-  and  technology-oriented  provisions 


42  U.S.C. 

§  1973ff(b)(6); 

42  U.S.C.  §  1973ff- 
4a(b) 


Report  on  effectiveness 
of  assistance  and  activities 
(§  1973ff-2b),  voter 
participation,  state- 
federal  cooperation,  use 
of  registration  assistance 
(§  1566a),  use  of  ballot- 


Analysis  of  implications  of  contact  with 
FVAP  resources  in  2012  postelection 
report  to  Congress  (FVAP,  2013b)b 
FVAP  website — in  particular,  the  link  to 
"Reports  to  Congress"  (www.fvap.gov/ 
reference/annualreports),  found  under 
"Reference/Reports"  then  "Reports" 


collection  and  ballot- 
delivery  procedures 
(§  1973ff-2a) 

42  U.S.C.  Develop  standards  for  data 

§  1 973ff (b)(1 1)  collection  and  storage  (with 

EAC  and  election  officials), 
which  also  implies  data 
storage 

42  U.S.C.  §  1973ff-7  May  establish  one  or 

more  pilot  programs  to 
test  feasibility  of  new 
technology 

42  U.S.C.  Serve  on  EAC  board  of 

§  15344(a)(14)  advisors  (applies  to  the 

director  of  FVAP) 


•  Work  ongoing 

•  FVAP  website — in  particular,  the  link 
to  "Surveys"  (www.fvap.gov/refer- 
ence/surveys),  found  under  "Reference/ 
Reports" 

•  FVAP  website — for  example,  the  link  to 
"Grants"  (www.fvap.gov/leo/grants), 
found  under  "Election  Officials" 


•  It  is  our  understanding  that  the  FVAP 
director  would  serve  on  this  board 
if  the  board  were  convened  and  the 
director  were  called  to  do  so 
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Table  4.1 — Continued 


Title  and  Section  or 
Subsection 

Summary  of  Requirement 

Corresponding  Activity  or  Location  of 
Information  About  the  Activity,  as  of 
November  2013 

Other  requirements 

42  U.S.C.  §  1973ff  note 

Shall  carry  out  electronic 
voting  demonstration 
project  but  may  delay 
implementation  of  project 
under  certain  circumstances 
(applies  to  Secretary  of 
Defense)0 

•  Research  and  analysis  to  support 
potential  implementation 

•  The  project  would  become  mandatory 
pending  EAC  notification  of  guidelines 
and  certification  of  assistance01 

42  U.S.C.  §  1973gg 

Promote  exercise  of 
right  to  vote  (applies  to 
federal,  state,  and  local 
governments) 

•  Carried  out  through  information  dis¬ 
semination  and  voter  outreach  pro¬ 
grams,  including  motivational  posters 

NOTE:  All  the  FVAP  URLs  listed  in  this  table  were  active  as  of  November  13,  2013,  but  most  or  all  are 
now  inactive  (stale)  because  of  a  substantial  redesign  of  the  FVAP  website.  We  present  the  URLs  as 
historical  data  points  to  substantiate  our  analysis  of  FVAP  activities.  Many  of  the  pages  historically 
present  at  those  URLs  can  be  viewed  using  the  Internet  Archive  Wayback  Machine. 
a  Technically,  the  provision  refers  to  "this  subchapter,"  which  tracks  to  "this  Title"  in  UOCAVA  and  is 
a  reference  to  Title  I  of  UOCAVA,  i.e.,  "Registration  and  Voting  by  Absent  Uniformed  Services  Voters 
and  Overseas  Voters  in  Elections  for  Federal  Office,"  as  codified  in  42  U.S.C.  §§1973ff  et  seq. 
b  See  the  discussion  of  sufficiency  under  "Through  the  Lens  of  Federal  Voting  Assistance  Program 
Activities." 

0  Implementation  would  have  triggered  an  additional  reporting  requirement  under  42  U.S.C.  §  1973ff 
note  H  (c). 

d  Section  593  of  the  National  Defense  Authorization  Act  for  Fiscal  Year  2015,  titled  "Repeal  of 
Electronic  Voting  Demonstration  Project,"  eliminated  this  requirement. 


e.g.,  relating  to  personnel  management  or  budgeting.7  For  this  analysis,  we  considered 
activities  that  stood  out  because  they  appeared  to  occupy  a  large  share  of  the  agency’s 
portfolio,  they  emerged  as  focal  points  in  the  logic-modeling  process,  or  stakeholders 
expressed  interest  in  them,  but  we  did  not  consider  all  FVAP  activities.8 

We  grouped  FVAP  activities  into  the  following  categories: 

•  Provide  VAO  training. 

•  Develop,  distribute,  collect,  and  deliver  ballot  materials. 

•  Engage  state  and  local  election  officials. 

•  Develop,  coordinate,  and  maintain  IT  systems  and  tools. 

•  Disseminate  information  and  conduct  outreach  to  voters. 


7  Recall  that  we  are  focusing  on  law  as  the  foundation  for  the  core,  given  the  potential  to  redraft  policy,  over 
time,  but  we  reference  policy  in  our  discussions  for  comparative  purposes. 

8  As  in  Chapter  Two,  we  touch  on  issues  of  VAO  training  and  research-related  activities  in  this  chapter,  but  we 
do  not  delve  deeply  into  either  because  we  explored  both  more  rigorously  in  phase  2  of  the  project. 
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•  Oversee  and  conduct  data  collection. 

•  Prepare  reports. 

Most  of  the  categories  draw  from  the  activity  streams  and  vocabulary  of  the 
benchmark  logic  model,  but,  in  some  cases,  such  as  election-official  engagement  and 
information  dissemination  and  outreach,  we  bundled  related  activities  within  or  across 
activity  streams.  Moreover,  in  the  case  of  developing,  distributing,  collecting,  and 
delivering  ballot  materials,  we  discuss  some  activities  that  drew  stakeholders’  attention 
but  that  do  not  appear  in  the  logic  model.  For  example,  collection  and  delivery  do  not 
feature  prominently  in  the  model  because  DoD  has  delegated  part,  if  not  all,  of  the 
responsibility  to  MPSA.  Table  4.2  summarizes  our  findings. 

In  the  sections  that  follow,  we  explain  the  reasoning  behind  our  assignments  of 
more  than  core  coverage,  less  than  core  coverage ,  and  unknown  in  Table  4.2.9 

Beyond  Core  Activities 

In  comparing  FVAP  activities  to  the  law,  we  identified  six  areas  in  which  FVAP  activi¬ 
ties  appeared  to  reach  beyond  the  core:  VAO  training,  the  website  and  portal,  legis¬ 
lative  initiatives,10  the  Voting  Assistance  Guide,  information  dissemination  and  out¬ 
reach,  and  data  collection.  However,  in  many  instances,  the  agency’s  activities  track  to 
paragraphs  in  DoDI  1000.04,  2012. 

Voting  Assistance  Officer  Training 

In  the  case  of  VAO  training,  which  FVAP  develops  and  offers  in  person  and  online,  the 
law  specifies,  “Commanders  at  all  levels  are  responsible  for  ensuring  that  unit  voting 
officers  are  trained  and  equipped  to  provide  information  and  assistance  to  members  of 
the  armed  forces  on  voting  matters,”11  but  does  not  specify  the  developer,  provider,  or 
modes  of  assistance.  In  that  sense,  there  are  no  specific,  direct  statutory  requirements — 
in  UOCAVA  or  elsewhere — for  FVAP  or  the  presidential  designee’s  involvement. 

To  operationalize  the  legislative  mandate  for  VAO  training,  DoDI  1000.04, 
2012,  calls  on  FVAP  to  “develop  and  deliver  multiple  types  of  training  materials  for 
use  by  IVA  offices,  IVAOs  [installation  VAOs],  UVAOs,  and  recruiters”  and  conduct 
voting  assistance  training  during  even-numbered  years  worldwide,  and  it  requires  that 


9  See  Table  4.1  for  corresponding  evidence  of  consistency. 

10  We  did  not  probe  this  issue  at  the  time  of  our  analysis  because  it  was  our  understanding  that  FVAP’s  “leg¬ 
islative  initiative”  activities  would  be  terminated,  which  they  were.  However,  we  note  that  the  law,  although 
once  referring  to  legislative  recommendations  (see  42  U.S.C.  §  1973ff  note,  now  expired,  “Governors’  Reports 
on  Implementation  of  Recommendations  for  Changes  in  State  Law  Made  under  the  Federal  Voting  Assistance 
Program”),  does  not,  at  present,  call  for  FVAP  or  the  presidential  designee  to  engage  with  states  on  legislative 
matters,  except  insomuch  as  it  is  making  the  states  aware  of  their  responsibilities  under  UOCAVA.  Yet,  FVAP 
continued  to  play  a  part  in  states’  legislative  discussions,  e.g.,  through  the  provision  of  testimony  at  hearings,  and 
was  described  by  stakeholders  as  a  legislative  advocate. 

11  Commanders  are  responsible  for  ensuring  training  under  10  U.S.C.  §  1566(f)(1). 
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Table  4.2 

Selected  Federal  Voting  Assistance  Program  Activities  in  Relation  to  Law 


More  Than 

Less  Than  Core 

Consistent 

Activity  Stream 

Core  Coverage 

Coverage 

with  Core 

Unknown 

Provide  VAO  training 

X 

Develop,  distribute,  collect,  and 
deliver  balloting  materials,  including 
cooperation  with  USPS 

X 

Engage  state  and  local  election 
officials3,  with  specific  reference  to 

Legislative  initiatives 

X 

Consulting  and  ensuring  awareness 

X 

Develop,  coordinate,  and  maintain  IT 
systems,  tools,  and  databases,  with 
specific  reference  to 

Website  and  portal 

X 

Online  assistants 

X 

Online  repository 

X 

Disseminate  information  and  conduct 
outreach  to  voters,  with  specific 
reference  to 

Notification 

X 

Information  and  procedural  outreach 

X 

Voting  Assistance  Guide 

X 

Other  outreach,  including  marketing 

X 

Oversee  and  conduct  data  collection, 
with  specific  reference  to  postelection 
and  other  surveys 

X 

X 

Prepare  reports  (annual,  quadrennial, 
and  other,  e.g.,  grant  programs) 

X 

SOURCES:  RAND  staff  analysis  based  on  logic  model-development  process,  conversations  with  FVAP 
stakeholders,  and  review  of  42  U.S.C.  §§1973ff  et  seq.  and  other  relevant  statutes. 

NOTE:  More  than  core  coverage  indicates  activities  that  appeared  to  reach  beyond  the  core.  Less  than 
core  coverage  indicates  an  apparent  gap  in  the  coverage  of  core  legal  requirements.  Consistent  with 
core  indicates  rough  consistency  with  core  requirements.  Unknown  indicates  indeterminate  or  lacking 
sufficient  information  to  draw  a  firm  conclusion  with  respect  to  one  or  more  dimensions  of  the  activity 
or  requirements  at  time  of  the  phase  1  analysis. 
a  Except  data  collection,  reporting,  and  grants. 


IVAOs  and  UVAOs  complete  FVAP  training  prior  to  assuming  duties.  Moreover,  the 
instruction  indirectly  calls  for  the  availability  of  both  in-person  and  online  training 
and  prefers  in-person  training.  It  requires  that  all  major  command  VAOs,  IVAOs,  and 
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UYAOs  attend  FVAP  voting  assistance  workshops;  asks  unit  commanders  to  provide 
funding  to  enable  UVAOs  to  attend  in-person  training  to  the  extent  practicable;  and 
references  online  training  as  an  alternative  for  VAOs  in  remote  locations. 

Responsive  to  DoDI  1000.04,  FVAP  develops  and  delivers  multiple  types  of  train¬ 
ing  materials,  for  use  in-person  and  online,  and  it  conducts  training  sessions  during 
even-numbered  years  worldwide.  In  the  2014  training  cycle,  FVAP  trainers  held  ses¬ 
sions  at  about  100  installations,  embassies,  and  consulates  and  delivered  training  to 
more  than  4,500  attendees.12  More  than  4,700  VAOs  took  the  online  training  in  a 
roughly  similar  period.  At  least  some  of  the  4,500  and  4,700  in-person  and  online 
trainees  might  have  taken  both  types  of  training.13 

Website  and  Portal 

Many  FVAP  activities,  including  VAO  training,  information  dissemination,  and  voter 
outreach,  make  use  of  the  website  and  portal.  FVAP  redesigned  its  website  in  200814 
and,  in  2010,  made  its  website  a  “portal,”15  with  the  intention  to  link  the  voter  directly 
to  online  state  and  local  jurisdiction  voting  assistance  services  (FVAP,  201  If,  p.  39).  In 
the  past  five  years,  FVAP’s  efforts  have  focused  on  the  deployment  and  enhancement 
of  direct  voting  assistance  tools,  such  as  the  online  assistants,  which  a  UOCAVA  voter 
can  use  to  obtain  information  about  voting  in  his  or  her  jurisdiction  and  walk  through 
the  process  of  completing  the  FPCA  and  FWAB.  In  addition,  FVAP  has  made  the 
information  published  in  the  Voting  Assistance  Guide  available  electronically  and  pro¬ 
vides  links  to  training  materials,  reports,  and  reference  materials. 

Title  42  U.S.C.  §  1973ff  contains  several  provisions  that  speak  directly  to  require¬ 
ments  for  IT  systems,  tools,  and  databases,  including  the  website,  the  portal,  and 
online  assistants.  The  law  also  calls  for  the  creation  of  portals  of  information  on  voter- 
registration  and  absentee-ballot  procedures;  the  establishment  of  a  system  of  voter  noti¬ 
fication,  using  the  military  Global  Network;  the  availability  and  maintenance  of  an 
online  repository  of  state  contact  information;  and  the  storage  of  data  on  the  number 
of  absentee  ballots  transmitted  and  received. 


12  FVAP  provided  the  data  on  training  sessions  and  trainees. 

13  During  the  training  sessions  that  we  observed  (Chapter  Six),  a  nontrivial  number  of  trainees  raised  their  hands 
when  the  trainer  asked  who  had  taken  the  online  training,  and  some  appeared  to  have  taken  it  recently. 

14  FVAP  undertook  another  major  redesign  of  its  website  in  2014,  subsequent  to  the  completion  of  the  analysis 
for  this  chapter.  For  that  reason,  many  of  the  links  and  URLs  for  FVAP  resources  and  information  that  we  say 
were  available  at  the  time  of  this  analysis  are  no  longer  available. 

15  Since  2010,  FVAP  has  initiated  development  of  a  comprehensive  informational  portal  and  content-management 
system  to  serve  as  a  resource  for  external  stakeholders  and  voters. 
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DoDI  1000.04,  2012 — and  current  practice — appeared  to  take  these  provisions, 
especially  those  relating  to  portals,  further: 

•  The  law  specifies  portals  of  information  on  voter-registration  and  absentee-ballot 
procedures,  but  the  instruction  specifies  a  portal  that  would  host  service-specific 
information  regarding  voting  assistance  programs;  contact  information  for  VAOs 
(installation,  major  command,  and  commissioned  units)  within  the  DoD  com¬ 
ponent;  procedures  to  order  voting  materials;  and  links  to  other  federal  and  state 
voting  websites. 

•  The  law  targets  the  portals  “to  inform  absent  uniformed  services  voters”  in  par¬ 
ticular,  but  the  instruction  only  implies  this  audience. 

•  The  instruction  also  calls  for  an  online  portal  to  collect  and  consolidate  voting- 
program  metrics,  but  neither  their  collection  nor  consolidation  is  required  in  law. 

To  the  extent  that  FVAP  activities  tended  to  track  the  provisions  of  the  instruc¬ 
tion  (DoDI  1000.04,  2012),  and  in  large  part  they  appeared  to  do  so,  the  agency’s 
activities  were  also  reaching  beyond  the  core. 

Voting  Assistance  Guide 

The  Voting  Assistance  Guide  for  2012-2013,  which  was  current  at  the  time  of  this 
analysis,  described  the  procedures  for  registering  to  vote,  requesting  a  ballot,  and  voting 
in  50  states,  the  District  of  Columbia,  and  four  territories,  with  contact  information. 
It  also  included  copies  of  the  FWAB,  FPCA,  and  electronic  transmission  cover  sheet. 

One  could  argue  that  the  2012-2013  Voting  Assistance  Guide  contributed  to  the 
fulfillment  of  a  range  of  statutory  requirements,  including  the  compilation  and  distri¬ 
bution  of  descriptive  material  on  state  absentee  registration  and  voting  procedures  and 
other  relevant  facts  (42  U.S.C.  §  1973ff[b][5])16  and  the  use  of  the  FWAB  (42  U.S.C. 
§  1973ff-2[a][2]).  The  latter  provides  voters  with  another  channel  of  access  to  that  form 
and  to  a  means  of  transmission. 

However,  in  serving  all  these  roles,  both  online  and  in  print,  the  Voting  Assis¬ 
tance  Guide  might  go  further  than  necessary.  Whereas  the  law  calls  for  FVAP  to  com¬ 
pile  and  distribute  descriptive  procedural  material  and  other  facts,  the  law  does  not 
require  that  FVAP  provide  this  information  in  a  bound  volume.  Nevertheless,  FVAP 
prepared — and  continues  to  prepare,  albeit  at  reduced  volume17 — a  printed  edition, 
which,  in  turn,  requires  updates  and  errata  sheets. 


16  Under  42  U.S.C.  §  1973ff(b)(5),  the  presidential  designee  shall  “compile  and  distribute  (A)  descriptive  mate¬ 
rial  on  State  absentee  registration  and  voting  procedures,  and  (B)  to  the  extent  practicable,  facts  relating  to  spe¬ 
cific  elections,  including  dates,  offices  involved,  and  the  text  of  ballot  questions.  .  .  .” 

1 '  It  is  our  understanding  that  FVAP  has  reduced  the  size  of  its  printing  runs  of  this  guide  and  is  prioritizing  the 
maintenance  and  upkeep  of  its  online  version. 
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Other  Outreach,  Including  Marketing 

FVAP  conducts  outreach  to  voters  through  many  channels,  including  its  website,  bulk 
email,  media,  partner  pass-through,  and  person-to-person  contact.18  FVAP  uses  bulk 
email  and  social  networking  to  share  absentee-voting  deadlines  and  procedures  and  to 
offer  resources  to  UOCAVA  voters  to  inform  and  educate  them  about  the  UOCAVA 
absentee-voting  process  (FVAP,  201  If,  p.  46).  In  addition,  FVAP  runs  PSAs  on  absen¬ 
tee  voting  using  television  and  radio  (e.g.,  the  American  Forces  Radio  and  Television 
Service  and  the  American  Forces  Network)  and  places  notices  in  private  and  military- 
focused  print  publications,  such  as  Stars  and  Stripes ;  the  Army  Times,  Navy  Times,  and 
Air  Force  Times-,  the  International  Herald  Tribune-,  and  other  overseas  publications. 
FVAP  also  provides  downloadable  posters  on  its  website,  for  use  by  VAOs  and  IVA 
offices,  on  military  installations.19 

The  law  requires  that  the  presidential  designee  provide  UOCAVA  voters,  espe¬ 
cially  uniformed-service  voters,  with  information  on  voting  procedures,  the  availability 
of  voting  assistance  and  resources,  state-level  contacts,  and,  to  the  extent  possible,  spe¬ 
cific  elections,  including  dates,  offices  involved,  and  the  text  of  ballot  questions.  In  some 
instances,  the  law  prescribes  the  development  or  use  of  a  particular  technology,  e.g., 
the  military  Global  Network  for  notification;  in  other  instances,  it  does  not.  Require¬ 
ments  pertaining  to  “descriptive  material  on  State  absentee  registration  and  voting 
procedures”  and  “facts  relating  to  specific  elections”  (42  U.S.C.  §  1973ff[b][5])  and  the 
“online  repository  of  State  contact  information”  (42  U.S.C.  §  1973ff-l[e][4])  and  the 
general  call  for  federal,  state,  and  local  governments  to  promote  the  exercise  of  the  right 
of  citizens  to  vote  (42  U.S.C.  §  1973gg)  apply  broadly  to  absent  uniformed-service 
voters  and  to  overseas- citizen  voters.20  The  rest  apply  only  to  absent  uniformed-service 
voters  or,  for  outreach  on  collection  and  delivery  procedures  (42  U.S.C.  §  1973ff-2a[c]), 
even  more  narrowly  to  absent  “overseas”  uniformed-service  voters. 

FVAP  appears  to  be  meeting  the  basic  informational  requirements  in  that  it 
undertakes  activities  relevant  to  each,  such  as  these: 

•  With  regard  to  voter  notification  (42  U.S.C.  §  1973ff-2b[a][2]),  FVAP  notifies 

military  members  of  voter-registration  information  and  resources  by  sending 


18  In  2010,  FVAP  engaged  a  contractor  to  develop  a  voting  assistance  communications  and  marketing  plan 
(FVAP,  201  If,  p.  46).  According  to  FVAP,  2 0 1 1  f,  p.  46,  the  goal  of  the  marketing  plan  was  to  expand  outreach 
to  voters,  improve  brand  recognition  ofFVAP.gov,  drive  users  to  online  tools,  and  raise  overall  awareness  of  key 
milestones  and  dates  for  voters  to  meet  in  order  to  successfully  cast  a  ballot. 

19  The  posters  were  located  on  FVAP’s  website  at  the  “Motivational  Posters”  tab  (formerly  www.fvap.gov/vao/ 
posters),  under  the  “Voting  Assistance  Officers”  heading.  We  note  that  the  URL  is  no  longer  valid,  though  a  snap¬ 
shot  of  the  content  formerly  there  was  available  through  the  Internet  Archive  Wayback  Machine  as  of  August  13, 
2015.  FVAP  now  refers  to  the  posters  as  informational  posters. 

20  The  requirement  for  a  technology-based  system,  under  42  U.S.C.  §  1973ff-2(a)(2),  also  applies  broadly. 
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email  blasts  using  the  military  Global  Network  90,  60,  and  30  days  prior  to  the 
general  election. 

•  With  regard  to  outreach  (42  U.S.C.  §  1973ff-2a[c];  10  U.S.C.  §  1566a[d]),  FVAP 
uses  social  media  channels,  such  as  Facebook  and  Twitter,  to  push  out  informa¬ 
tion  on  how  to  register  to  vote  and  request  absentee  ballots  (42  U.S.C.  §  1973ff- 
2a[c]).21 

•  With  regard  to  compiling  and  distributing  descriptive  materials  on  state  absen¬ 
tee  registration  and  voting  procedures,  FVAP  provides  relevant  information  and 
materials  in  the  online  and  print  versions  of  the  Voting  Assistance  Guide.  Voters 
can  also  obtain  state-specific  information  through  the  online  assistants  on  the 
FVAP  website. 

The  law  on  UOCAVA  voting  says  very  little  about  outreach  as  a  means  to  moti¬ 
vate  voting;  rather,  it  speaks  to  “outreach”  in  terms  of  information  about  procedures 
(42  U.S.C.  §  1973ff-2a[c])  and  the  availability  of  assistance  through  voter  assistance 
offices  (10  U.S.C.  §  1566a[d]),  targeting  uniformed-service  voters.  Title  10  U.S.C. 
§  1566(i)(3),  although  not  directed  at  FVAP,  takes  a  slightly  broader  approach  in  requir¬ 
ing  that,  “[w]here  practicable,  a  special  day  or  days  shall  be  designated  at  each  military 
installation  for  the  purpose  of  informing  members  of  the  armed  forces  and  their  depen¬ 
dents  of  election  timing,  registration  requirements,  and  voting  procedures.”22  Only  in 
42  U.S.C.  §  1973gg,  which  speaks  of  an  imperative  to  promote  the  exercise  of  the  right 
of  citizens  to  vote,  does  the  law  edge  nearer  to  a  call  for  motivational  outreach. 

By  contrast,  DoDI  1000.04,  2012,  refers  to  notification  “[a]s  a  component  of  a 
comprehensive  marketing  and  voter  education  initiative”  (enclosure  3,  f  j,  p.  10);  it  also 
calls  on  the  services  to  develop  a  component-wide  communication  plan  to  “encourage 
voting  participation”  (enclosure  4,  f  2k,  p.  15),  among  other  things. 

Data  Collection,  Including  Surveys 

FVAP  undertakes  data  collection  through  regularly  scheduled  surveys  and  other 
means.  In  addition  to  conducting  surveys,  FVAP  tracks  website  and  tool  usage;  works 
with  external  providers,  such  as  Google  Analytics;  records  indicators  of  IVA  offices’ 
compliance  with  DoD  policy;  and  obtains  quarterly  reports  from  IVAOs.  In  the  dis- 


21  In  the  case  of  outreach  regarding  ballot  delivery  and  collection,  DoDI  1000.04,  2012,  assigns  responsibility  to 
MPSA.  Enclosure  4,  5  2ad(2),  p.  18,  calls  on  MPSA  to  “[d]evelop  an  outreach  plan  to  inform  overseas  uniformed 
services  voters  regarding  the  ballot  collection  and  delivery  service  to  be  implemented  prior  to  each  general  elec¬ 
tion  for  Federal  office.” 

22  As  noted  in  Table  4.1,  “absentee-voter  weeks”  (designated  weeks  with  installation-based  activities  that  create 
awareness  of  absentee  voting  and  provide  information  about  and  resources  for  absentee  voting) — and  email  alerts 
and  other  voter-oriented  outreach — might  also  be  contributing  to  the  fulfillment  of  the  core  requirement  for 
procedures  to  promote  and  expand  the  use  of  the  FWAB.  Similarly,  FVAP’s  conference  attendance  and  other 
outreach  directed  toward  state  and  local  election  officials  might  be  serving  to  improve  the  officials’  awareness  of 
their  requirements,  which  is  also  a  core  requirement. 
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cussion  that  follows,  we  focus  on  the  set  of  routine  postelection  surveys,  which  FVAP 
conducts  in  coordination  with  DMDC. 

According  to  FVAP,23  these  surveys  were  intended  to  do  the  following: 

•  Determine  UOCAVA  citizens’  participation  in  the  electoral  process. 

•  Assess  the  impact  of  FVAP’s  efforts  to  simplify  and  ease  the  process  of  voting 
absentee. 

•  Evaluate  other  progress  made  to  facilitate  absentee  voting  participation. 

•  Identify  any  remaining  obstacles  to  voting  by  these  citizens. 

•  Meet  legislative  requirements  for  reporting  on  the  effectiveness  of  assistance, 
including  statistical  analyses  of  UOCAVA  voter  participation. 

Separately,  FVAP  has  surveyed  ADM  personnel,  ADM  spouses,  overseas  citizens,24 
UVAOs,  State  Department  VAOs,  and  local  election  officials.25 

In  Appendix  C,  we  present  data  on  survey  response  rates,  some  of  which  have 
been  quite  low,  and  discuss  possible  reasons,  including  the  length  and  frequency  of  the 
surveys. 

We  categorized  the  activity  as  “more  than  core  coverage”  in  Table  4.2,  in  part, 
because  the  law  does  not  call  for  FVAP  to  conduct  surveys.  Although  it  requires  a 
variety  of  activities  that  would  explicitly  or  implicitly  involve  data  collection,  it  might 
not  necessitate  surveys  that  cover  all  the  aforementioned  recipient  categories  in  such 
depth,  breadth,  or  frequency.  FVAP  must  report  on  effectiveness,  voter  registration 
and  participation,  state-federal  cooperation,  utilization  of  voting  assistance,  and  ballot 
collection  and  delivery;  develop  and  implement  online  tools,  consisting  of  the  portals, 
repository,  and  online  assistants;  compile  and  distribute  descriptive  material  on  state 
absentee  registration  and  voting  procedures  and  facts  relating  to  specific  elections;  and 
store  state  data  collected  by  EAC  with  input  from  FVAP.  In  the  case  of  voter  participa¬ 
tion  (42  U.S.C.  §  1973ff[b][6]),  the  quadrennial  analysis  must  be  “statistical.” 

Title  10  U.S.C.  §  1566(g)  contains  the  only  explicit  legislative  reference  to  the 
conduct  of  surveys.26  It  calls  on  the  Secretary  of  Defense  to  conduct  periodic  surveys 
of  all  overseas  locations  and  vessels  at  sea  with  military  units  responsible  for  collecting 
mail  for  return  shipment  to  the  United  States  and  all  port  facilities  in  the  United  States 
and  overseas  where  military-related  mail  is  collected  for  shipment  to  overseas  locations 


23  The  first  four  items  are  based  on  information  that  was  and  still  is  provided  on  the  FVAP  website  (FVAP, 
undated  [b]);  the  last  one  is  based  on  42  U.S.C.  §  1973ff(b)(6)  and  communication  with  FVAP. 

24  According  to  FVAP,  the  overseas-citizen  surveys  have  been  discontinued  for  methodological  reasons  and  until 
such  time  as  FVAP  can  reasonably  identify  the  overseas-citizen  voter  population. 

25  FVAP  and  EAC  were  merging  their  respective  quantitative  surveys  at  the  time  of  this  analysis. 

26  MPSA  conducted  an  online  end-of-year  survey  for  the  Military  Postal  System’s  absentee  ballot-handling  pro¬ 
cedures  and  performance  during  the  2010  general  election.  See  MPSA,  undated. 
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or  to  the  United  States.  The  surveys  are  to  be  conducted  for  the  purpose  of  determin¬ 
ing  whether  voting  materials  are  awaiting  shipment  at  those  locations  and,  if  so,  for 
how  long. 

It  remains  to  be  determined  whether  surveying  is  the  right  approach  to  fulfilling 
the  various  statutory  requirements  set  out  above,  but,  as  a  step  toward  making  that 
determination,  one  can  ask  whether  the  postelection  survey  questions  relate — at  least 
conceptually — to  those  requirements  and,  if  so,  whether  they  are  necessary  or  appro¬ 
priate  to  meeting  the  requirements.  On  that  basis,  we  note  that  the  surveys  are  long 
and,  depending  on  how  one  interprets  the  requirement  for  reporting  on  effectiveness,27 
some  questions  might  lack  a  direct  statutory  connection.28  The  data  in  Table  2.1  in 
Chapter  Two  suggest  that  relatively  few  survey  questions  directly  support  FVAP’s  stat¬ 
utory  reporting  requirements;  nevertheless,  they  might  serve  other  institutional  pur¬ 
poses,  e.g.,  for  internal  planning  and  strategy  development.  As  addressed  in  the  related 
RAND  project  on  metrics  and  measures,  surveys  can  provide  insight  to  what  is  or  is 
not  working,  on  which  the  program  can  build. 

In  addition,  we  found  that  FVAP  might  be  collecting  similar  information  from 
more  than  one  source.  In  some  instances,  e.g.,  relating  to  voter  participation,  topical 
coverage  appears  to  be  duplicative  across  surveys  and  other  data-collection  mecha¬ 
nisms,  such  as  IVAO  quarterly  reports.29  This  type  of  duplication  might  be  inten¬ 
tional  for  purposes  of  cross-referencing,  triangulation,  and  validation,  but  we  do  know 
whether  such  processes  have  occurred  or  proven  insightful  for  reporting  or  managerial 
purposes. 

As  in  each  of  the  prior  discussions,  FVAP’s  activities  appeared  to  be  at  least  roughly 
in  line  with  the  text  of  DoDI  1000.04,  2012;  for  example,  the  instruction  mandates 
surveys  and  identifies  a  set  of  survey  populations  that  includes  ADM  personnel  and 
dependents,  overseas  U.S.  civilians,  VAOs,  and  election  officials. 

Finally,  we  note  that,  although  we  categorized  this  activity  as  reaching  beyond 
the  core,  it  is  still  possible  that  some  elements  of  the  data  collection  did  not  fully  cover 
the  core.  Thus,  lacking  sufficient  information  to  make  a  final  determination,  we  have 
also  placed  a  mark  in  the  “Unknown”  column.  The  surveys  might  be  excessive  in  some 


27  The  wording  of  the  effectiveness  requirement  in  42  U.S.C.  §  1973ff-4a(b)(l)  could  support  at  least  two  differ¬ 
ent  approaches  to  reporting,  one  much  narrower  than  the  other,  which  we  note  later  in  this  chapter. 

28  FVAP  cites  satisfaction  results  in  the  discussion  of  federal  voting  assistance  activities  in  the  2012  postelection 
report  to  Congress  (see,  for  example,  FVAP,  2013b,  pp.  27-28).  By  all  appearances,  the  citations  to  the  satisfac¬ 
tion  results  seem  to  equate  satisfaction  with  effectiveness. 

29  Similar  data  are  also  collected  from  state  and  local  election  officials,  and  their  response  rate  is  higher.  For 
example,  weighted  response  rates  of  local  election  officials  in  the  2010  postelection  survey  was  53  percent,  whereas 
weighted  response  rates  for  active-duty  military  service  members  and  active-duty  military  spouses  were  15  per¬ 
cent  and  14  percent,  respectively  (FVAP,  201  If).  Response  rates  were  not  provided  in  FVAP’s  2012  postelection 
report  to  Congress  (FVAP,  2013b).  Data  on  voter  registration  and  participation  by  absent  uniformed-service 
voters  are  also  collected  through  the  quarterly  reports  submitted  by  IVAOs  and  the  EAC  local  election-official 
survey,  which  is  merging  with  the  FVAP  survey. 
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regards  but  still  insufficient  or  inappropriately  focused  to  meet  reporting  requirements, 
including  those  on  effectiveness.  We  return  to  this  issue  in  the  context  of  reporting  but 
suggest  it  as  a  topic  for  FVAP’s  further  exploration. 

Candidates  for  Augmentation 

In  comparing  FVAP  activities  to  the  law,  we  identified  three  areas  in  which  FVAP 
might  need  to  augment  its  efforts  to  align  its  activities  with  statutory  requirements: 
developing,  distributing,  collecting,  and  delivering  balloting  materials;  engaging  state 
and  local  election  officials;  and  preparing  reports.  With  regard  to  the  first  area,  we 
learned  that  some  of  the  elements  in  question,  e.g.,  ballot  collection  and  delivery,  had 
been  delegated,  in  part,  to  MPSA,30  but  we  could  not  account  fully  for  other  elements 
related  to  the  use  of  the  FWAB. 

Procedures  to  Promote  and  Expand  the  Use  of  the  Federal  Write-In  Absentee  Ballot 

FVAP  plays  many  parts  in  developing,  facilitating,  and  encouraging  the  use  of  the 
FWAB.  For  example,  it  shepherds  the  process  of  revising  and  updating  the  form;  pro¬ 
vides  tools,  such  as  the  online  assistants  and  Electronic  Transmission  Service  (ETS); 
conducts  awareness  campaigns  and  other  outreach;  and  helps  to  distribute  hard  copies, 
if  needed. 

The  law  requires  FVAP  to  adopt  procedures  to  promote  and  expand  the  use  of  the 
FWAB  as  a  backup  measure  to  vote  in  elections  for  federal  office  (42  U.S.C.  §  1973ff- 
2  [a]  [2]  [A]),  but  the  meaning  of  procedure  is  open  to  interpretation.  The  requirement 
for  such  procedures  appeared  to  be  separate  and  distinct  from  the  requirement  for  the 
utilization  of  technology  to  implement  a  system  for  voters  to  obtain  populated  ballots 
and  print  and  mail  them.  FVAP  appeared  to  be  meeting  the  technology  requirement 
with  the  online  assistants,  but  whether  its  other  efforts  constituted  procedures  was 
unclear.31  On  that  basis,  we  marked  “Unknown”  in  Table  4.2  but  would  suggest  that 
FVAP  clarify  its  interpretation  of  the  requirement  in  future  guidance. 

Consulting  State  and  Local  Election  Officials  and  Ensuring  Their  Awareness 

FVAP  engages  with  state  and  local  election  officials  in  at  least  two  distinct  modes:  one 
involving  the  provision  of  assistance  and  the  other  involving  the  collection  and  use 
of  information.  FVAP  has  assisted  election  officials  by  providing  online  training  and 
guidance,  sending  email  alerts,  funding  research  grants,  participating  in  conferences, 
conducting  other  local  outreach,  and  making  direct  (person-to-person)  contact.  In 


30  FVAP  notes  that  it  plays  a  part  in  collection  and  delivery  and  that  its  role  with  MPSA  is  more  than  advisory; 
for  example,  in  2010,  it  funded  MPSA  to  purchase  the  express-mail  scanners  to  expedite  mail.  FVAP  also  coor¬ 
dinated  with  USPS  the  delivery  days  and  times  for  express-mail  package  times.  According  to  FVAP,  it  also  plays 
a  part  in  conducting  outreach  to  absent  overseas  uniformed-service  voters  on  procedures  for  the  collection  and 
delivery  of  marked  absentee  ballots  (42  U.S.C.  §  1973ff-2a[c]). 

31  More  recently,  FVAP  has  also  begun  work  with  the  Council  of  State  Governments  (CSC)  to  discuss  the  use  of 
forms  as  UOCAVA  voter-registration  applications. 
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addition,  FVAP,  in  consultation  with  DOJ,  considers  and  responds  to  states’  requests 
for  waivers  of  certain  legal  requirements  under  the  MOVE  Act.  As  an  informational 
conduit,  FVAP  also  engages  with  election  officials  through  data  collection,  compila¬ 
tion,  storage,  reporting,  and  distribution.  For  purposes  of  this  discussion,  we  focused 
on  engagement  as  assistance  and  not  on  data  collection. 

The  law  (42  U.S.C.  §  1973ff[b][l])  includes  a  requirement  for  FVAP  to  consult 
state  and  local  election  officials  in  carrying  out  UOCAVA32  and  ensure  their  awareness 
of  the  requirements  of  UOCAVA.  We  considered  FVAP’s  activities  in  two  parts:  first, 
with  respect  to  consulting  and,  second,  with  respect  to  ensuring  awareness. 

The  evidence  on  consulting  was  mixed.  FVAP  was  surveying  local  election  offi¬ 
cials  on  their  use  of  and  satisfaction  with  FVAP  products33  and  engaging  with  officials 
at  conferences,  such  as  those  of  the  Election  Center  and  NASED,  but  was  not  reaching 
out  regularly  to  understand  their  perspectives  on  UOCAVA  more  generally.34 

FVAP’s  efforts  to  ensure  awareness  merited  closer  consideration.  The  law  set  a 
high  bar  for  attainment  when  it  required  FVAP  to  ensure — rather  than  promote  or 
encourage — awareness.  Much  of  FVAP’s  state-  and  local-oriented  activity  might  have 
promoted  or  encouraged  awareness,  but  it  was  unclear  whether  it  ensured  awareness. 
FVAP  was  providing  election  officials  with  online  training  and  guidance,35  but  we 
found  mixed  evidence  of  the  use  of  these  resources.  FVAP’s  conference  participa¬ 
tion  and  other  local  outreach  might  have  been  raising  officials’  awareness  of  the  legal 
requirements  but  was  not  uniformly  geared  toward  that  purpose.36  Email  alerts  were 
targeted  more  explicitly,  e.g.,  as  reminders  of  deadlines,  but  did  not  cover  the  full  spec¬ 
trum  of  requirements.  In  June  2013,  FVAP  began  publishing  a  newsletter  for  election 
officials  (FVAP,  2013d).  The  newsletter,  if  continued,  could  provide  an  opportunity  for 
FVAP  to  contribute  to  awareness  on  a  more  deliberate  and  regular  basis. 


32  The  U.S.  Code  refers  to  “this  subchapter,”  which  tracks  to  “this  Title”  in  Pub.  L.  99-410  and  is  a  reference  to 
Title  I  of  UOCAVA,  i.e.,  “Registration  and  Voting  by  Absent  Uniformed  Services  Voters  and  Overseas  Voters  in 
Elections  for  Federal  Office,”  as  codified  across  42  U.S.C.  §§  1973ff  et  seq. 

33  See,  for  example,  the  2012  Post-Election  Qualitative  Voting  Survey  of  Local  Election  Officials  (DMDC, 
2013a,  2013b). 

34  Subsequent  to  the  phase  1  analysis,  FVAP  embarked  on  a  relationship  with  CSC,  potentially  reflective  of  a 
more-systematic  approach  to  engaging  with  the  states.  We  address  this  development  in  the  presentation  of  evi¬ 
dence  of  change  in  Chapter  Seven. 

35  This  training  and  guidance  was  available  on  the  FVAP  website  under  the  “Election  Official”  tab  (formerly 
www.fvap.gov/leo/index)  during  our  period  of  analysis.  This  URL  is  no  longer  valid,  but  the  image  that  was  once 
there  was  viewable  using  the  Internet  Archive  Wayback  Machine  as  of  August  13,  2015. 

36  It  is  our  understanding  that,  subsequent  to  the  phase  1  analysis,  FVAP  began  developing  a  checklist  of  require¬ 
ments  for  election  officials  but  had  not  yet  completed  the  task  at  the  time  of  this  writing. 
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Congressional  Reports  on  Effectiveness  and  Other  Matters 

Table  4.3  lists  the  laws  related  to  FVAP  reporting  requirements  and  the  reports  that 
FVAP  has  submitted  in  response  to  each  requirement  according  to  the  FYAP  website. 
Appendix  C  provides  background  on  the  different  reports. 

The  law  sets  out  reporting  requirements  for  FVAP  in  42  U.S.C.  §  1973ff(b)(6) 
and  §  1973ff-4a,  the  latter  of  which  also  references  42  U.S.C.  §  1973ff-2b.  In  1986, 
§  102(b)  of  Pub.  L.  99-410,  i.e.,  UOCAVA,  introduced  42  U.S.C.  §  1973ff(b)(6);  in 
2009,  §  586  and  §  583  of  Pub.  L.  111-84,  which  included  the  MOVE  Act,  added 

Table  4.3 

Reporting  Requirements  and  Related  Reports 


Report  Submitted  or  Created  by  FVAP  in  Apparent 
Statutory  Requirement  for  FVAP  Reporting  Response  to  Statutory  Requirement 


42  U.S.C.  §  1973ff-4a(a)  Report  on  status  of 
implementation  and  assessment  of  programs 


42  U.S.C.  §  1973ff-4a(b)  Annual  report  on 
effectiveness  of  activities  and  utilization  of 
certain  procedures* * 3 


42  U.S.C.  §  1 973ff (b)(6)  Duties  of  presidential 
designee  pertaining  to  quadrennial  reportb 


42  U.S.C.  §  1973ff-7  Technology  pilot  program 


42  U.S.C.  §  1973ff  note  H  (c) 


•  Report  on  the  Status  and  Implementation  of 
Military  and  Overseas  Voter  Empowerment 
Act  Programs  (FVAP,  2011c) 

•  2012  Post-Election  Report  to  Congress  (FVAP, 
2013b) 

•  2011  Annual  Report  on  the  Effectiveness  of 
Activities  and  Utilization  of  Certain  Procedures 
(FVAP,  2012) 

•  2010  Post  Election  Survey  Report  to  Congress 
(FVAP,  201  If) 

•  2009  Annual  Report  on  the  Effectiveness  of 
Activities  and  Utilization  of  Certain  Procedures 
(FVAP,  2011b) 

•  2012  Post-Election  Report  to  Congress  (FVAP, 
2013b) 

•  Eighteenth  Report:  2008  Post  Election  Survey 
Report  (FVAP,  201  Id) 

•  The  Federal  Voting  Assistance  Program:  Seven¬ 
teenth  Report  (FVAP,  2005b) 

•  The  Federal  Voting  Assistance  Program:  The 
16th  Report  (FVAP,  2001) 

•  The  Federal  Voting  Assistance  Program:  Fif¬ 
teenth  Report  (FVAP,  1997) 

•  2010  Electronic  Voting  Support  Wizard  (EVSW) 
Technology  Pilot  Program  Report  to  Congress 
(FVAP,  2013e)c 

•  Implementation  of  the  demonstration  project 
would  have  triggered  an  additional  reporting 
requirement. 


SOURCES:  RAND  staff  analysis  based  on  content  of  42  U.S.C.  §§  1973ff  et  seq.  and  the  above-referenced 

FVAP  reports  to  Congress. 

3  The  MOVE  Act  introduced  this  reporting  requirement  in  October  2009.  The  corresponding  reports 
listed  in  the  table  postdate  that  legislation. 

b  This  reporting  requirement  has  been  in  place  in  a  similar  but  not  identical  form  since  1986. 

c  FVAP  now  refers  to  the  specific  tools  that  are  directly  available  and  supported  by  FVAP  through  FVAP. 
gov,  including  wizard-like  tools,  as  online  assistants. 
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42  U.S.C.  §  1973ff-4a  and  §  1973ff-2b.  Some  elements  of  both  sets  of  provisions — the 
provisions  that  date  back  to  1986  and  the  newer  provisions — are  open  to  interpreta¬ 
tion,  but  we  focus  on  the  newer  ones. 

The  scope  of  the  reporting  requirement  for  an  assessment  of  the  effectiveness  of 
assistance  activities  (42  U.S.C.  §  1973ff-4a[b]  [1])  can  be  read  narrowly  or  broadly,37  but 
one  can  still  ask  whether  FVAP’s  reports  speak  to  the  effectiveness  of  such  activities — 
be  they  few  or  many — or  to  any  other  statutory  reporting  requirements. 

We  found  that  the  annual  and  postelection  reports  featured  sections  that  related 
topically  to  most,  if  not  all,  of  the  requirements  but  that  the  material  contained  in  each 
section  did  not  uniformly  meet  the  requirements.  In  the  report  section  titled  “FVAP 
Activities,”  which  appeared  to  be  intended  to  address  the  effectiveness  requirement, 
most  of  the  indicators  spoke  to  outputs,  usage,  and  satisfaction,  none  of  which  gets  at 
effectiveness.  In  the  parlance  of  the  logic  model,  the  information  on  usage  and  satisfac¬ 
tion  has  the  advantage  of  venturing  into  the  domain  of  intermediate  effects  but  does 
not  get  as  far  as  outcomes  or  mission  attainment.  Although  it  appears  as  if  FVAP — or 
the  survey  developer — was  equating  customer  satisfaction  and  program  effectiveness, 
these  are  not  equivalent  concepts.38 

Coming  closer  to  singling  out  effectiveness,  FVAP  (FVAP,  2013b,  p.  19)  reported 
a  statistical  relationship  between  contact  with  DoD  voting  assistance  resources  and  the 
likelihood  of  an  active-duty  member  voting  and  returning  his  or  her  ballot: 

The  most  significant  finding  indicates  that  of  the  active  duty  members  who  inter¬ 
acted  with  one  of  these  resources  and  received  an  absentee  ballot,  86%  voted 
and  returned  their  absentee  ballots;  whereas  for  those  active  duty  members  who 
received  an  absentee  ballot,  but  never  interacted  with  one  of  the  resources,  only 
80%  voted  and  returned  their  ballots. 

Although  it  is  unclear  whether  the  analysis  uncovered  a  causal  link  between  interac¬ 
tion  with  FVAP  resources  and  successful  voting  outcomes — voters  might  interact  with 
those  resources  because  they  are  more  interested  in  or  intent  on  voting  and,  hence, 


37  In  42  U.S.C.  §  1973ff-4a(b)(l),  the  phrase  “including  the  activities  and  actions  of  the  Federal  Voting  Assis¬ 
tance  Program”  appears  to  be  tied  to  the  opening  line,  “An  assessment  of  the  effectiveness  of  activities  carried  out 
under  42  USC  §1973tf-2b  of  this  title.”  On  that  basis,  the  contents  of  42  U.S.C.  §  1973ff-2b  might  establish  the 
bounds  of  the  reporting  requirement,  but  those  bounds  might  still  allow  the  possibility  of  different  approaches 
to  annual  reporting.  One  approach  could  focus  on  duties  that  are  set  out  in  42  U.S.C.  §  1973ff-2b(a),  involving 
the  development  of  online  portals  and  the  establishment  of  a  voter-notification  program.  Another  approach  could 
be  more  expansive  and  take  into  consideration  everything  covered  under  42  U.S.C.  §  1973ff-2b(a)  and  §  1973ff- 
2b(b),  which  confirms  the  presidential  designee’s  duties  and  responsibilities  under  directive  or  regulation.  It  is  also 
possible  that  the  term  Federal  Voting  Assistance  Program  refers  in  this  context  not  just  to  FVAP,  the  agency,  but  to 
all  federal  voting  assistance  offered  under  DoD  or  other  federal  auspices. 

38  See  Greenfield  and  Camm,  2005,  for  a  discussion  of  the  relationship  between  customer  satisfaction,  as  embod¬ 
ied  in  quality  ratings,  and  contract  spending  on  service  provision. 
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more  likely  to  succeed  in  voting  (FYAP,  2014a) — it  suggests  the  potential  value  of 
increasing  voters’  awareness  of  F  VAP’s  resources.39 


Concluding  Remarks  on  Activities,  Requirements,  and  Findings 

Our  approach  in  this  chapter  consisted  of  a  crosswalk  among  law,  policy,  and  FVAP 
activities.  We  examined  the  relationship  between  FVAP  activities  and  core  require¬ 
ments  found  mostly  in  UOCAVA.  To  a  lesser  extent,  we  also  considered  FVAP  activi¬ 
ties  and  law  in  relation  to  provisions  of  policy  found  in  DoDI  1000.04,  2012.  Our 
analysis  did  not  reveal  chasms  in  FVAP’s  coverage  of  the  core;  nevertheless,  it  sug¬ 
gested  that  FVAP  might  need  to  do  more  to  ensure  state  and  local  officials’  awareness 
of  requirements;  to  assess,  in  its  reports,  the  effectiveness  of  voting  assistance;  and,  pos¬ 
sibly,  to  develop  procedures  to  promote  and  expand  the  use  of  the  FWAB.40 

With  regard  to  our  assessments  of  some  activities,  such  as  VAO  training,41  as 
reaching  beyond  the  core,  we  note  again  that  these  are  areas  in  which  FVAP  might 
have  some  flexibility  in  its  approach,  yet  still  serve  the  voting  community  construc¬ 
tively.  In  the  case  of  the  printed  and  bound  Voting  Assistance  Guide,  stakeholders 
across  communities,  including  those  serving  overseas-citizen  voters,  spoke  out  for 
its  benefits.  Similarly,  we  recall  that  stakeholders  supported  the  augmentation  of  the 
FVAP  website  and  suggested  new  and  additional  features  for  voters  and  other  users. 
Finally,  we  note  that  outreach  might  be  supporting  core  requirements  other  than  those 
specifically  referring  to  outreach,  including  those  pertaining  to  the  use  of  the  FWAB 
and  to  engagement  with  election  officials,  and  serve  an  important  role  in  promoting 
voter  awareness.42 

Fiaving  completed  our  look  at  requirements  and  determined  that  FVAP  might 
have  some  latitude  to  chart  its  path  in  the  activities  that  it  undertakes  to  support  the 
core,  meet  more-general  requirements,  and  meet  other  institutional  needs,  we  were 
able  to  address  what  it  might  mean  to  do  so.  Returning  to  the  Venn  diagram,  we  con¬ 
sidered  the  relative  positions  of  the  ovals.  In  Figure  4.2,  we  suggest  that  FVAP  would 
need  to  institute  enough  change  to  cover  the  law  oval  fully  but,  upon  meeting  the  core 
requirements  found  in  UOCAVA  and  elsewhere,  it  could  then  consider  alterations  to 
the  positions — and  size  and  contents — of  the  other  ovals. 


39  pvAP  recognizes  that  the  positive  relationship  does  not  establish  causality  and  that  individual  characteristics, 
such  as  interest  and  intent,  might  play  a  part  in  the  result.  It  has  written  about  these  issues  in  a  recent  research 
note.  See  FVAP,  2014a,  p.  6. 

40  We  address  more-recent  gains  in  these  areas  in  Chapter  Seven. 

41  We  address  this  issue  at  length  in  Chapter  Six. 

42  We  discuss  FVAP’s  more-recent  evidence-based  targeting  of  outreach  in  Chapter  Seven. 
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Figure  4.2 

What  Does  It  Mean  to  Chart  a  Path? 


RAND  RR882-4.2 


In  addition,  our  analysis  revealed  two  organizational  challenges.  First,  the  pro¬ 
visions  of  law  and  the  activities  that  pertain  to  them  do  not  occupy  discrete,  harmo¬ 
nized  organizational  “buckets.”  For  example,  two  core  activities — ensuring  awareness 
and  conducting  outreach — involve  combinations  of  assistance,  communications,  and 
IT,  which,  in  turn,  are  organizationally  separate  and  interdependent.  The  difficulty  of 
bucketing  suggested  the  importance  of  organizational  coherence,  something  that  we 
called  out  in  Chapter  Two  as  potentially  lacking.  Second,  a  shortfall  in  reporting  on 
effectiveness  might  reflect  an  underlying  lack  of  awareness  of  effectiveness.  Along  such 
lines,  our  initial  review  of  FVAP’s  reporting  on  effectiveness  and  our  conversations 
with  stakeholders  suggested  that  FVAP  and  the  larger  voting  community  would  ben¬ 
efit  from  a  better  understanding  of  the  relationship  between  the  costs  and  benefits  of 
its  activities,  both  in  setting  and  in  implementing  policy. 

In  Chapter  Five,  we  discuss  the  recommendations  that  flowed  from  each  of  the 
foregoing  chapters,  including  those  speaking  to  mission  ambiguity  and  to  the  size, 
position,  and  content  of  the  activity  oval. 


CHAPTER  FIVE 


Preliminary  Recommendations  and  Initial  Guidance 


At  the  close  of  phase  1  of  the  project,  we  introduced  a  set  of  recommendations  and 
guidance  as  preliminary  and  initial  because  the  project  was  not  yet  complete.  In  later 
phases,  we  continued  to  explore  FVAP’s  relationships  with  its  partners  and  customers, 
including  VAOs  and  IVA  offices,  and  its  approach  to  research. 

Nevertheless,  the  focal  points,  themes,  and  other  findings  that  emerged  from  each 
step,  i.e.,  logic-model  development,  stakeholder  outreach,  and  the  assessment  of  the 
relationship  between  FVAP  activities  and  requirements,  strongly  suggested  that  FVAP, 
its  stakeholders,  and  the  law  were  each  seeing  the  world  differently. 

We  can  summarize  their  perspectives  as  follows: 

•  FVAP  saw  itself,  in  large  part,  as  benefiting  UOCAVA  voters  through  intermedi¬ 
aries,  such  as  VAOs,  election  officials,  and  NGOs. 

•  Stakeholders  did  not  see  a  clear  path  from  FVAP  to  voting  success  and  were 
uncertain  about  what  FVAP  was  doing  or  why. 

•  The  law  saw  FVAP  doing  more  than  it  must  in  some  regards  and  possibly  less 
than  it  must  in  others. 

We  interpreted  the  differences  among  the  three  perspectives  as  signaling  substan¬ 
tive  disconnects.  If,  for  example,  FVAP  saw  itself  as  operating  through  intermediar¬ 
ies,  but  those  same  intermediaries  could  not  see  the  path  from  FVAP  to  them  or  to 
UOCAVA  voters,  it  seemed  likely  that  something  was  amiss. 

This  chapter  recasts  the  findings  from  each  of  the  preceding  chapters  as  actionable 
recommendations  and  guidance  for  FVAP  leadership. 

However,  we  note  that,  by  the  time  we  formally  delivered  these  recommendations 
to  FVAP,  the  agency  had  already  begun  to  act  on  them.  This  was  possible  because  we 
shared  our  findings  as  they  emerged,  to  generate  debate  and  discussion.  Nevertheless, 
we  recount  them  here  to  provide  a  record  of  this  phase  of  the  project  and  a  means  of 
gauging  the  changes  that  we  report  in  Chapter  Seven.  Absent  a  clear  understanding  of 
where  FVAP  started,  it  might  be  difficult  to  appreciate  how  much  and  how  quickly  it 
has  changed. 
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Become  "One  FVAP" 

To  become  one  FVAP,  the  agency  would  need  to  come  to  terms  with  its  mission,  inte¬ 
grate  and  shore  up  its  operations  to  support  that  mission,  and  clarify  and  sharpen  its 
message. 

Come  to  Terms  with  Its  Mission 

To  start,  we  suggested  that  FVAP  hone  its  mission  in  a  manner  that  is  consistent  with 
law  and  policy  and  establish  a  common,  internally  shared  understanding  of  the  agen¬ 
cy’s  purpose  and  priorities.  Merely  agreeing  to  a  mission  statement,  potentially  subject 
to  myriad  interpretations,  would  not  be  enough.  Coming  to  terms  would  entail  estab¬ 
lishing  a  set  of  primary  functions,  consisting  of  core,  supporting,  and  institutional 
activities,  which,  in  turn,  should  derive  from  an  evidence-based  analysis  of  require¬ 
ments,  needs,  and  potential  or  actual  effectiveness. 

Our  engagement  with  stakeholders  strongly  suggested  that  FVAP  is  doing  what 
it  does  best  when  it  offers  direct  assistance  and  responds  to  the  expressed  needs  of  its 
partners  and  customers.  In  recognition  of  the  latter,  FVAP  might  consider  engaging 
directly  with  its  stakeholders  in  coming  to  terms  with  its  mission.  Although  it  cannot 
satisfy  everyone’s  interests — some  stakeholder  interests  are  inherently  conflicting — 
FVAP  can  draw  its  partners  and  customers  into  the  process  and  perhaps  build  stronger 
relationships  with  them. 

As  general  guidance,  we  suggested  simplicity.  We  recommended  a  handful  of  ini¬ 
tial  steps  to  better  ensure  that  state  and  local  election  officials  are  aware  of  UOCAVA 
requirements,  such  as  the  following: 

•  Develop  a  checklist  of  state  and  local  requirements  with  links  to  relevant  guid¬ 
ance  documents — and  produce  relevant  guidance  if  none  exists. 

•  Reach  out  to  election  officials  on  a  routine,  scheduled  basis. 

•  Take  a  vendor  perspective  at  state  and  local  conferences,  and  maintain  a  table  or 
booth  with  necessary  supplies,  including  the  checklist  and  corresponding  guid¬ 
ance. 

Lastly,  we  noted  that  some  of  FVAP’s  challenges  with  mission  and  priorities  might 
be  a  consequence  of  leadership  churn  and  attendant  policy  changes.  To  the  extent  that 
FVAP  is  entering  an  era  of  institutional  stability,  it  might  be  better  able  to  address  these 
issues. 

Integrate  and  Shore  Up  Operations 

Operational  integration  could  occur  through  the  reorganization  and  pooling  of 
resources  into  fewer  divisions  or  through  actions  taken  to  strengthen  the  connective 
tissues  of  the  organization.  In  terms  of  the  logic  model,  the  former  would  imply  having 
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fewer  boxes  and  activity  streams,  and  the  latter  would  mean  drawing  firmer  arrows 
between  them. 

The  choice  of  approach  would  depend  on  decisions  about  the  agency’s  mission 
and  priorities,  including  the  prominence  of  motivation  and  FVAP’s  role  in  training  and 
research,  but  some  integrative  steps  could  be  taken  regardless.  For  example,  the  agency 
could  begin  a  staff-wide  conversation  about  the  research  needs  of  each  activity  stream 
and  start  to  identify  means  to  channel  the  results  of  research  to  those  streams. 

Regardless  of  the  agency’s  choice  of  organizational  structure,  we  recommended 
that  FVAP  establish  clear  lines  of  accountability  to  legislative  mandates;  that  is,  it 
should  be  able  to  trace  each  provision  of  law  to  a  person  with  primary  responsibility 
for  related  activities.  The  activities  might  cut  across  organizational  lines — as  we  noted 
earlier,  the  law  does  not  lend  itself  to  organizational  bucketing — but  it  should  be  pos¬ 
sible  to  identify  a  lead  for  each  provision. 

As  a  related  matter,  we  suggested  that  FVAP  address  apparent  top-heaviness.  To 
start,  the  director  might  consider  what  role  each  leader  is  serving;  how  that  role  tracks 
to  the  mission,  priorities,  and  institutional  needs  of  the  agency;  and  whether  the  role  is 
duplicative  of  a  role  filled  elsewhere  in  the  organization.  Similarly,  FVAP  might  benefit 
from  considering  the  distribution  of  staff  members  across  activity  streams. 

Both  to  promote  integration,  by  increasing  staff  understanding  of  the  interrelat¬ 
edness  of  FVAP  activities,  and  to  better  address  staffing  constraints,  we  recommended 
that  FVAP  consider  cross-training  personnel.1  FVAP  might  also  explore  opportunities 
to  adopt  common  strategies  and  shared  methods  across  activity  streams,  as  in  the  cases 
of  building  and  maintaining  relationships,  developing  and  modifying  training  materi¬ 
als  and  guidance,  and  identifying  best  practices.  This  is  not  to  say  that  personnel  or 
tools  can  or  should  be  viewed  as  strictly  interchangeable,  but  to  recognize  opportuni¬ 
ties  to  leverage  resources. 

To  address  remaining  capacity  and  capability  deficits,  FVAP  would  need  to  assess 
its  functional  needs  (e.g.,  skills  and  tools)  in  relation  to  its  priorities,  identify  gaps,  and 
take  actions  to  fill  them.  For  example,  FVAP  might  find  that  it  needs  just  enough  ana¬ 
lytical  ability  to  establish  its  research  priorities,  frame  research  questions,  make  use  of 
findings,  and  oversee  external  contracts.  We  recommended  that  FVAP  consider  pro¬ 
fessional  development  as  a  means  to  fill  some  of  the  gaps.2  Professional  development 
appeared  to  be  available  to  FVAP  staff  on  an  ad  hoc  basis,  but  FVAP  could  identify 


1  We  knew  that  election-official  assistance  staff  had  begun  to  accompany  voter  assistance  staff  during  VAO 
training  sessions  and  viewed  this  as  a  step  in  the  direction  of  cross-training. 

2  If  seeking  to  become  a  standard  bearer,  as  some  stakeholders  suggested,  FVAP  might  encourage  professional 
development  as  a  step  toward  staff  members  becoming  best  in  class  in  their  respective  fields. 
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needs,  both  institutional  and  individual,  and  opportunities  more  comprehensively  and 
systematically.3 

We  also  suggested  that  FYAP  consider  converting  term  appointments  that 
appeared  to  support  primary  functions  to  permanent  appointments.  This  staffing  deci¬ 
sion  would  require  consideration  of  what  constitutes  a  primary  function,  which  would, 
in  turn,  require  that  FVAP  address  issues  of  mission  and  priorities. 

Sharpen  and  Clarify  the  Message 

In  whatever  way  FVAP  chose  to  orient  its  mission  and  priorities,  the  agency  would 
also  need  to  harmonize  its  messaging — across  leadership  and  staff  at  all  levels  of  the 
organization  and  across  outputs,  including  online  guidance  and  training  materials, 
public  presentations,  congressional  reports,  PSAs,  and  other  communications.  Beyond 
a  common,  shared  understanding  of  the  agency’s  mission  and  priorities,  FVAP  leader¬ 
ship  and  staff  would  benefit  from  a  common,  shared  vocabulary  for  describing  them. 
We  recommended  that  FVAP’s  communication  group  take  responsibility  for  the  con¬ 
sistency  of  language  across  divisions  and  outputs  and,  perhaps,  create  communication 
materials  for  staff  use  to  support  consistent  messaging.  We  address  the  issue  of  mes¬ 
saging  again  in  Chapter  Eight  but  note  that  logic  models  can  provide  a  foundation 
for  internal  and  external  communication  and  that  the  final,  robust  logic  model  that 
emerged  from  this  project  could  serve  in  that  role. 


Build  Trust  and  Strengthen  Relationships 

We  also  suggested  that  FVAP  attend  to  its  relationships  with  stakeholders.  Stronger, 
two-way  relationships  could  enable  FVAP  to  use  its  own  resources  more  effectively, 
e.g.,  by  spending  less  time  fighting  fires,  better  leveraging  the  resources  of  its  partners, 
and,  ultimately,  better  serving  its  customers.  We  suggested  a  fundamental  transforma¬ 
tion  in  FVAP’s  relationships,  requiring  a  reconceptualization  of  the  agency’s  approach 
to  its  stakeholders  and,  more  concretely,  behavioral  changes  around  communication 
and  transparency.  Our  views  on  these  issues  largely  echoed  the  sentiments  of  FVAP’s 
stakeholders. 

Work  with  Partners  and  Serve  Customers 

Here,  we  emphasized  the  importance  of  valuing — and  treating — partners  and  custom¬ 
ers  as  active  participants  in  the  voting  system,  not  as  passive  actors.  To  do  so,  FVAP 
would  need  to  recognize  the  different  circumstances  of  its  stakeholders  and  the  impli¬ 
cations  of  those  differences.  We  illustrate  by  categorizing  UOCAVA  voters  as  either 


3  These  recommendations  are  consistent  with  OPM  guidelines  on  employee  development  and  training  and  the 
use  of  an  individual  development  plan.  For  more  information  on  OPM  guidelines,  see  OPM,  2005;  for  more 
information  on  the  use  of  an  individual  development  plan,  see  OPM,  undated. 
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uniformed  or  overseas  citizens.  In  our  discussion  of  the  benchmark  logic  model,  we 
observed  that  FVAP  is  not  especially  well-positioned  to  reach  out  directly  to  overseas 
voters  who  are  not  members  of  the  uniformed  services,  and,  in  our  conversations  with 
stakeholders,  we  heard  concerns  about  the  effort  directed  toward  them  and  the  extent 
to  which  DoD,  if  not  FVAP,  held  interest  in  them.  On  that  basis,  FVAP  might  seek 
to  reach  overseas  civilians  indirectly,  through  alternative  channels — that  is,  to  culti¬ 
vate  its  relationships  with  potential  intermediaries.  Overseas  NGOs,  for  example,  have 
established  a  combination  of  grassroots,  person-to-person,  and  online  connections  with 
overseas-citizen  voters;  if  working  collaboratively  with  those  NGOs,  FVAP  might  be 
able  to  draw  on  their  connections  to  provide  better  service  to  voters  whom  it  cannot 
reach  otherwise.4 

Communicate  Better  and  More  Regularly 

Stakeholders  were  nearly  unanimous  in  asking  for  both  more  and  better  communica¬ 
tion  from  FVAP  and  in  their  portrayals  of  what  that  would  mean.  From  their  perspec¬ 
tives,  FVAP  would  need  to  reach  out  more  often  and  more  regularly,  listen  to  what 
its  stakeholders  have  to  say  when  it  engages  with  them,  and  respond  to  them,  even  if 
only  to  say  “we  cannot  do  what  you  asked  us  to  do  .  .  .  and  here  is  why.”  Moreover, 
even  in  broadcast  mode,  stakeholders  indicated  opportunities  that  we  believed  merited 
consideration — for  example,  bolstering  the  VAO,  election-official,  and  reporting  com¬ 
ponents  of  the  website  to  be  more  user-friendly  and  routinizing  outreach  so  that  it  is 
more  predictable,  even  as  a  touch-base  mechanism,  e.g.,  “we  are  sending  this  newsletter 
to  keep  you  up  to  date  on  recent  improvements  to  .  .  .”  or  “we  are  contacting  you  to 
find  out  whether  you  have  any  concerns  or  questions  about.  .  .  .”5 

As  a  related  matter,  we  also  took  the  part  of  FVAP’s  stakeholders  in  suggesting 
that  FVAP  continue  to  work  diligently  to  avoid  the  appearance  of  partisanship,  politi¬ 
cization,  or  salesmanship.  Recalling  the  stakeholders’  observation  that  FVAP’s  out¬ 
reach  sometimes  appeared  to  be  driven  too  much  by  personal  agenda  and  not  enough 
by  the  agency’s  mission,  we  recommended  close  adherence  to  the  mission,  once  decided 
and  articulated. 

Operate  as  Openly  as  Possible 

Recognizing  that  FVAP  is  serving  a  community  that  extends  well  beyond  the  military 
but  operates  within  a  national  security  institution  (DoD)  and  must  abide  by  its  rules, 
we  recommended  that  FVAP  operate  as  openly  as  possible  and  do  what  it  can  to  share 
what  it  can,  when  it  can.  FVAP  faces  an  inevitable  tension  around  information  dis¬ 
semination:  In  the  security  community,  caution  is  a  priority;  in  the  voting  community, 
openness  is  more  the  norm.  FVAP  cannot  eliminate  this  tension,  but  it  can  do  more  to 


4  Some  stakeholders  offered  specific  suggestions  that  we  forwarded  to  FVAP  separately. 

’  See  the  earlier  comments  on  state  and  local  engagement. 
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bridge  the  divide.  For  example,  it  can  post  data  sooner,  post  criteria  for  making  deci¬ 
sions,  and  let  stakeholders  know  when  change  is  in  the  works,  so  that  they  can  prepare 
for  it  and,  if  possible,  play  a  part  in  it. 

FVAP  might  not  be  able  to  satisfy  all  of  its  stakeholders,  given  their  sometimes- 
differing  views,  but  it  can,  through  increased  openness,  do  more  to  make  its  case  for 
why  it  does  what  it  does,  when  it  does  it.  If  the  agency  can  say  clearly  “our  purpose 
is  A”;  “our  priorities  are  X,  Y,  and  Z”;  and  “our  activities  are  based  on  the  follow¬ 
ing  evidence-based  analysis  .  .  .  ,”  it  might  be  able  to  argue  more  persuasively  for  its 
approaches  to  assistance. 


Embrace  a  Culture  and  Principles  of  Effectiveness 

We  finished  with  an  overarching  recommendation  for  FVAP  to  adopt,  internalize,  and 
apply  a  culture  and  principles  of  effectiveness  throughout  its  operations,  beginning 
with  its  work  on  its  mission,  priorities,  and  activities.  We  suggested  that  FVAP  take  up 
the  call  for  effectiveness  found  in  the  congressional  reporting  requirements  as  a  way  of 
doing  business — that  is,  as  the  underlying  premise  of  policy  and  actions  on  a  day-to- 
day  basis.  More  concretely,  we  suggested  that  FVAP  engage  routinely  in  benefit-cost 
assessment  (BCA),  be  it  quantitative  or  qualitative,  in  developing  and  implementing 
projects,  assessing  the  merits  of  proposed  activities,  and  as  a  way  of  thinking  about 
daily  tasks.6  We  were  not  asking  FVAP  to  conduct  BCA  simply  to  meet  basic  admin¬ 
istrative  requirements,  common  to  all  federal  agencies,  but  to  think  of  BCA  as  a  lens 
through  which  to  view  its  entire  operation.  Adopting  a  BCA-like  mind-set  could  lead 
to  a  more-rigorous  and  more-thorough  consideration  of  the  implications  of  policy  and 
actions  for  FVAP  and  for  the  voting  community — particularly  if  FVAP  were  to  take  a 
broad,  systemic  view  of  voting  assistance  and  consider  the  benefits  and  costs  of  activ¬ 
ities  throughout  the  community.  Flowever,  taking  a  broader  and  more  analytically 
oriented  approach  might  also  require  a  better  understanding  of  the  voting  assistance 
system  and  additional  analytical  means. 


Concluding  Remarks  on  Recommendations  and  Guidance 

We  started  the  discussion  of  phase  1  of  this  project  by  describing  our  analytical 
approach,  consisting  of  logic  modeling,  stakeholder  outreach,  and  a  requirements 
assessment.  At  the  outset,  we  noted  that  the  process  of  developing  the  benchmark  logic 
model  would  enable  us  to  see  the  agency  through  the  agency’s  eyes  and  that  the  next 


6  BCA  is  just  one  approach  to  understanding  effectiveness,  but  it  offers  the  advantages  of  well-accepted  and 
well-documented  methods  and  tools.  See  OMB,  2003.  If  FVAP  needs  to  identify  measures  or  cannot  readily 
quantify  benefits,  a  logic  model,  as  described  in  Chapter  Two  and  Appendix  A,  can  provide  insight. 
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two  approaches  would  supplement  the  model  by  enabling  us  to  see  FVAP  through  the 
eyes  of  its  stakeholders  and  in  terms  of  core  requirements,  respectively.  In  effect,  the 
second  and  third  views  would  provide  us  with  information  that  would  help  us  to  fill 
out,  test,  and  validate  the  model. 

We  found  that  the  three  perspectives  did  not  line  up.  FVAP  saw  itself  as  operat¬ 
ing  through  intermediaries  to  the  ultimate  benefit  of  UOCAVA  voters,  specifically  to 
“votes  cast,  received,  and  counted,”  but  its  stakeholders  did  not  share  that  view.  The 
differences  among  perspectives  appeared  to  be  a  problem  of  substance.  We  were  hope¬ 
ful  that  the  preliminary  recommendations  and  initial  guidance  could  help  FVAP  to 
both  strengthen  the  agency  and  reconcile  the  differences  in  perspectives.  In  a  series  of 
meetings  with  FVAP  leadership  and  staff  and  an  all-hands  presentation,  we  suggested 
that  FVAP  begin  by  solidifying  its  mission  and  resolving  issues  of  purpose  and  priori¬ 
ties  and  expand  its  efforts,  from  there.7 


7  In  taking  this  approach,  FVAP  would  be  charting  a  path  that  would  build  on  previous  efforts  to  strengthen  the 
organization  and  on  the  core  values  once  posted  at  the  FVAP  website.  According  to  those  values,  FVAP  should 
be  customer-focused,  proactive,  data-driven,  transparent,  and  efficient. 


CHAPTER  SIX 


The  Federal  Voting  Assistance  Program's  Engagement  with 
Its  Operating  Environment 


In  phase  2,  we  examined  how  FVAP  engages  with  its  operating  environment1  through 
partnerships,  research,  and  other  activities  and  identified  opportunities  for  it  to  do  so 
more  effectively  in  the  future.2  We  continued  to  probe  the  points  of  connectivity  in 
the  logic  model,  but  we  adopted  a  broader  analytical  lens  than  previously,  i.e.,  one  of  a 
larger  voting  assistance  system.3  In  effect,  FVAP  shifted  from  center  stage  and  became 
one  of  many  service  providers.  On  that  basis,  we  considered  the  agency’s  engagement 
within  the  system  in  two  ways. 

First,  we  considered  FVAP’s  approach  to  managing  and  disseminating  findings 
from  research.  FVAP  lacked  analytical  capabilities  and  capacities  (Chapter  Two),  some 
of  which  could  be  filled  through  professional  development  (Chapter  Five).  It  also  lacked 
a  method  for  selecting  projects,  leading  them,  and  channeling  their  results  to  advance 
its  mission.  To  address  the  gap  in  methods,  we  began  adapting  a  set  of  tools — a  check¬ 
list  and  worksheet,  based  largely  on  principles  of  BCA  and  risk  assessment — from 
existing  frameworks.  As  our  work  progressed,  the  general  applicability  of  the  tools 
became  apparent,  and,  together  with  FVAP,  we  concluded  that  they  merited  reframing 
as  tools  for  managing  projects  and  operations  more  generally.  We  present  the  tools  in 
Appendix  D. 

Second,  we  considered  FVAP’s  engagement  with  VAOs  (military  and  civilian) 
and  IVA  offices  as  conduits  of  voting  assistance  to  UOCAVA  voters.  We  took  a  deep 
dive  into  FVAP’s  assistance  to  military  installations,  including  the  IVA  offices,  and 
applied  the  newly  developed  tools  to  the  VAO  training  program.  Although  concentrat¬ 
ing  on  FVAP’s  work  with  the  military,  which  occupies  much  of  the  law  on  UOCAVA 


1  In  Chapter  One,  we  defined  operating  environment  as  including  customers,  partners,  and  other  stakeholders; 
related  organizations;  and  supporting  infrastructure  and  technology. 

2  See  Chapter  One  for  an  overview  of  analytical  challenges,  including  concurrent,  interdependent  institutional 
change  and  proposed  legislative  changes,  in  the  form  of  the  Safeguarding  Elections  for  Our  Nation’s  Troops 
Through  Reforms  and  Improvements  Act. 

3  As  we  discuss  below,  we  use  this  term  to  refer  to  the  various  providers  of  voting  assistance  to  UOCAVA  voters 
and  the  ways  in  which  they  relate  to  each  other,  e.g.,  as  a  loosely  formed  network,  not  to  a  formal  system. 
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voting  (Chapter  Four)  and  the  agency’s  resources,  we  also  considered  FVAP’s  work 
with  State  Departmenr  officials. 

To  identify  opportunities  to  streamline  and  strengthen  FVAP’s  engagement  activ¬ 
ities,  including  VAO  training,  we  set  the  following  goals: 

•  Examine  training  and  support  needs,  from  the  different  perspectives  of  FVAP, 
VAOs,  IVA  offices,  and  the  broader  voting  and  election  community. 

•  Consider  the  benefits,  costs,  and  risks  of  different  modes  of  product  and  service 
delivery. 

•  Identify  means  to  improve  connections  between  training  and  other  support  activ¬ 
ities  and  customer  and  partner  needs,  legislative  mandates,  and  other  institutional 
needs,  in  a  manner  that  is  consistent  with  the  assessment  of  costs,  benefits,  and 
risk.4 

Providing  a  bridge  between  phases  1  and  2,  we  also  began  work  with  FVAP  on 
the  development  and  refinement  of  the  robust  logic  model.  That  work  drew  heavily 
from  the  recommendations  and  guidance  outlined  in  Chapter  Five,  e.g.,  by  incorporat¬ 
ing  ideas  for  agency-wide  integration. 

The  remainder  of  this  chapter  proceeds  as  follows.  To  start,  it  describes  our 
approach  to  scoping  the  voting  assistance  system  and  identifying  the  needs  of  its  par¬ 
ticipants;  then,  it  characterizes  the  system  and  its  needs  and  considers  the  implications 
for  FVAP’s  engagement.  Next,  it  summarizes  our  Endings  on  the  VAO  training  pro¬ 
gram;  lastly,  it  discusses  the  key  elements  of  the  rearticulated  logic  model. 


Our  Approach  to  Scoping  the  System 

In  phase  2,  we  considered  how  FVAP  might  contribute  most  effectively  to  the  voting 
assistance  system  as  a  whole  and  by  parts.  We  focused  on  the  parts  pertaining  to  mili¬ 
tary  installations,  given  their  importance  to  UOCAVA,  the  MOVE  Act,  and  FVAP’s 
resourcing,  but,  in  our  consideration  of  VAO  training,  we  also  included  State  Depart¬ 
ment  officials.  We  did  not  focus  on  FVAP’s  engagement  with  election  officials,  but  they 
featured  prominently  in  our  characterization  of  the  voting  assistance  system  because 
of  their  centrality  to  voting.  Similarly,  we  did  not  focus  on  NGOs  but  include  them  in 
our  depiction  of  the  system.5 


4  We  did  not  conduct  a  full-scale  evaluation  of  FVAP’s  training  activities. 

5  To  do  so,  we  relied  heavily  on  information  gleaned  from  our  conversations  with  NGOs  and  explorations  of 
their  websites  and  product  offerings. 
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The  broader  lens  of  phase  2  required  a  more  expansive  evidentiary  base,  hence,  a 
need  to  reexplore  legal  requirements  relating  to  YAOs  and  IYA  offices  and  to  examine 
survey  and  training  evaluation  results,  others’  research  findings,  and  ground  truth. 

To  obtain  that  ground  truth,  we  met  with  VAOs  and  IVA  office  staff;  conducted 
site  visits;  and  observed  VAO  training  sessions  in  the  Washington,  D.C.,  metropolitan 
area,  California,  and  Colorado  at  Marine  Corps,  Navy,  Air  Force,  Army,  and  joint 
installations.  In  addition,  we  sat  in  on  a  remote  training  session  conducted  with  a 
consular  office  in  Athens,  Greece.  Our  site  selections  were  strategic  insomuch  as  we 
attempted  to  cover  each  of  the  services  and  a  variety  of  operational  and  administrative 
settings,  with  offices  led  by  ADM  personnel  and  civilians,  mindful  of  the  possibility 
of  cultural  differences  across  services  and  between  military  and  civilian  settings.  We 
observed  training  events  for  each  service  with  a  single  trainer  to  hold  trainer  effects 
constant  but  visited  additional  sites  with  different  trainers.  Table  6.1  chronicles  our  site 
visits  and  training  observations. 


Table  6.1 

Site  Visits  and  On-Site  Training  Observations 


Installation 

Service  Branch 

Training  Date 

Number  of  Trainees 

IVAO  Status 

Joint  Base  Myer- 
Henderson  Hall 

Joint 

n/a 

n/a 

Civilian 

Marine  Corps  Base 
Camp  Pendleton 

Marine  Corps 

January  14,  2014 

84 

Civilian 

Marine  Corps  Air 
Station  Miramar 

Marine  Corps 

n/a 

n/a 

Civilian 

Naval  Base 

Coronado 

Navy 

January  15,  2014 

13 

ADM 

Naval  Base  San 

Diego 

Navy 

January  16,  2014 

4 

ADM 

Marine  Corps  Base 
Quantico 

Marine  Corps 

March  18,  2014 

9 

ADM 

Fort  Belvoir 

Army 

March  18,  2014 

8 

Civilian 

Fort  Carson 

Army 

March  25,  2014 

206 

ADM 

Peterson  Air  Force 
Base3 

Air  Force 

March  26,  2014 

78 

Civilian 

SOURCES:  RAND  staff  analysis  and  FVAP  attendance  data. 

NOTE:  We  also  sat  in  on  a  remote  training  event  conducted  with  a  consular  office  in  Athens,  Greece,  on 
January  24,  2014.  n/a  =  not  applicable. 

a  The  training  was  held  at  Peterson  Air  Force  Base,  but  it  was  also  attended  by  Schriever  Air  Force  Base 
personnel,  including  the  IVAO. 
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In  our  travels,  we  focused  on  activities  and  operating  conditions  in  and  around 
the  IYA  offices  and  training  venues,  be  they  classrooms  or  auditoriums,  reflecting  on 
the  following: 

•  the  layouts  and  locations  of  I VA  offices,  the  use  of  the  offices,  and  a  range  of  instal¬ 
lation  parameters,  including  the  chain  of  command  for  voting  assistance,  the 
status  of  VAO  assignments — be  they  short-  or  long-term,  primary  or  ancillary — 
and  the  rate  of  turnover  among  VAOs.  Other  topics  of  interest  included  the  VAO 
or  IVA  office  manager’s  location  in  the  system  and  the  relationships  among  them, 
the  level  and  form  of  their  interactions  with  FVAP,  and  their  training  needs. 

•  the  physical  setup  of  the  training  venue;  the  level  of  participant  engagement; 
the  method,  e.g.,  structure  and  style,  of  training;  and  the  duration,  content,  and 
emphasis  of  training.  While  seeking  to  understand  the  system,  we  were  also  seek¬ 
ing  to  learn  whether  one  mode  of  delivery  (in  person,  online,  or  hybrid)6  would 
be  most  efficacious  and  whether  the  content  of  the  training  might  lend  itself  to  a 
particular  mode. 

The  reality  of  our  exploration  was  not  as  neatly  compartmentalized  as  these 
paragraphs  suggest.  In  visiting  the  IVA  offices,  we  learned  about  more  than  just  the 
IVA  offices;  in  observing  training,  we  learned  about  more  than  just  training.  Taken 
together,  our  consideration  of  the  IVA  offices  and  the  training  program  contributed 
greatly  to  our  understanding  of  the  configuration  and  functioning  of  the  voting  assis¬ 
tance  system,  to  the  needs  of  the  system,  and  to  FVAP’s  roles  and  responsibilities  in 
relation  to  that  system. 

As  part  of  the  scoping  exercise,  we  charted  a  UOCAVA  voter’s  access  to  assistance 
in  the  voting  process  and  derived  a  map  of  the  system. 


The  Federal  Voting  Assistance  Program's  Place  in  the  Voting 
Assistance  System 

In  phase  1,  we  found  that  FVAP’s  business  model  was  largely  indirect,  was  not  always 
effective  in  its  engagement  with  stakeholders,  and  allowed  room  for  realignment  of 
activities.  The  approach  was  indirect  in  that  FVAP  was  enabling  others,  military  and 
civilian,  to  better  serve  voters  and  often  operating  through  others,  such  as  VAOs,  to 
reach  voters,  create  awareness  among  voters,  and  disseminate  information  to  voters. 
FVAP  had  means  of  direct  engagement,  including  the  website  and  call  center,  but,  as  a 
very  small  agency  tasked  with  administering  a  law  that  serves  millions  of  voters,  it  did 
not  and  could  not  go  it  alone. 


6 


Hybrid  training  makes  use  of  combination  of  online  and  in-person  training. 
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The  work  of  phase  2  validated  that  description,  but  the  additional  evidence,  par¬ 
ticularly  the  appeal  to  ground  truth,  enabled  a  fuller  depiction,  both  literal  and  figura¬ 
tive,  of  the  voting  assistance  system. 

Figure  6.1  charts  a  UOCAVA  voter’s  opportunities  to  obtain  assistance  as  he  or 
she  moves  through  the  voting  process.  The  resulting  map  implies  the  contours  of  a 
system  of  voting  assistance  that  includes  federal,  state,  local,  and  private-sector  (NGO) 
service  providers  and  suggests  possible  paths  to  a  successful  voting  outcome.  The  map 
calls  out  the  voter’s  decision  to  obtain  assistance  and  the  options  available  at  each  junc¬ 
ture,  but  it  also  references  passive  assistance,  in  the  form  of  social  media,  PSAs,  emails, 
and  other  notifications.  (The  latter  forms  of  assistance  are  marked  in  the  figure  with 
blue  stars.)  The  map  also  distinguishes  among  different  types  of  UOCAVA  voters.  It 
indicates  whether  a  service  provider  or  particular  form  of  assistance  is  available  primar¬ 
ily  to  military  or  overseas-citizen  absentee  voters.  Moreover,  it  designates  one  starting 
point  for  first-time  UOCAVA  voters,  who  might  not  be  aware  of  their  status  and  rights 
as  UOCAVA  voters,  and  another  starting  point  for  returning  UOCAVA  voters,  who 
have  made  it  over  the  initial  hurdle  of  awareness. 

The  map  is  not  intended  to  provide  a  complete  picture  of  either  the  voting  process 
(even  though  it  tracks  steps  in  that  process)7  or  the  service-level  programs  that  house 
the  installation-based  assistance.8  However,  in  pegging  assistance  to  a  voter’s  experi¬ 
ence  of  the  voting  process,  which  is  itself  a  reasonably  well-defined  system,  it  might 
give  voting  assistance  the  appearance  of  more  form  and  structure  than  it  merits. 

Through  this  lens,  we  found  that  the  system  consists  of  many  potential  provid¬ 
ers  of  voting  assistance  and  points  of  service  (physical  or  virtual)  to  which  UOCAVA 
voters  can  turn  for  help  in  the  voting  process.  For  example,  an  absentee  ADM  voter 
might  approach  a  VAO  with  questions  about  his  or  her  voting  jurisdiction;  alterna¬ 
tively,  the  voter  might  reach  out  directly  to  a  state  or  local  election  official  or  go  to  an 
NGO’s  website.  In  this  context,  FVAP  is  one  provider  of  assistance  among  many. 

In  Figure  6.2,  we  parse  the  system  to  consider  it  both  as  a  whole  and  in  terms  of 
the  institutions,  such  as  FVAP,  that  function  within  that  system  and  the  people,  such  as 
VAOs,  who  act  on  behalf  of  those  institutions — or  themselves.  We  found  this  parsing 
to  be  helpful  both  as  a  reminder  of  the  sometimes-disparate  voices  within  the  system 
and  as  a  mechanism  for  better  understanding  the  needs  of  the  system  and  FVAP’s  role 
in  relation  to  those  needs.  Figure  6.2  also  incorporates  the  overarching  legal-policy 
environment  and,  by  extension,  the  requirements  or  needs  of  law  and  policy. 


7  For  example,  it  does  not  separately  depict  the  process  by  which  a  voter  would  determine  his  or  her  residency — 
this  is  implicit  in  “seeks  to  register” — or  all  the  means  by  which  a  voter  can  transmit  his  or  her  forms  or  ballot. 
On  that  basis,  it  omits  some  leading  actors  in  the  voting  process,  such  MPSA  and  USPS,  that  feature  prominently 
in  transmission  but  do  not  serve  as  assistance  providers  in  this  context. 

8  Similarly,  it  is  not  intended  to  chart  the  voting  assistance  programs  of  the  U.S.  Coast  Guard,  the  U.S.  Depart¬ 
ment  of  State,  or  the  U.S.  Public  Health  Service. 


Figure  6.1 

Voting  Assistance  Opportunities  and  Paths  to  Success  for  Uniformed  and  Overseas  Citizens  Absentee  Voting  Act  Voters 
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Figure  6.2 

The  Voting  Assistance  System  as  a  Whole  and  in  Parts 
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SOURCE:  RAND  staff  analysis  with  input  from  FVAP  staff. 


RAND  RR882-6.2 


Individuals 


UOCAVA  voter 
(ADM,  Dependent, 
overseas  citizen) 


Election  officials 


FVAP  (staff) 


UVAO 

(military) 


IVAO 

(military) 


IVA  office  (staff) 
(military) 


VAO 

(civilian) 


NGO  (staff) 


The  Federal  Voting  Assistance  Program's  Engagement  with  Its  Operating  Environment  73 


74  The  Federal  Voting  Assistance  Program  and  the  Road  Ahead 


Above  all,  we  observed  that  UOCAVA  voters  have  access  to  many  different  pro¬ 
viders,  through  many  different  points  of  service.  In  addition,  we  found  the  following: 

•  Law,  policy,  and  the  market  shape  the  system. 

•  Connectivity  and  coordination  vary  across  the  system. 

•  The  system  and  its  parts  need  knowledge,  capability,  and  capacity. 

•  FVAP,  in  particular,  faces  substantial  functional  limitations  in  the  system,  occu¬ 
pies  a  unique  position  in  the  system,  and  can  use  training  to  leverage  its  position 
in  the  system. 

We  address  each  point  in  the  following  sections. 

Law,  Policy,  and  the  Market  Shape  the  System 

Law,  policy,  and  the  market  each  play  a  part  in  establishing  the  elements  of  the  voting 
assistance  system,  which  cross  the  boundaries  of  the  public  (federal,  state,  or  local) 
sector  and  private  sector  (e.g.,  NGOs).  The  law  (e.g.,  42  U.S.C.  §  1973ff,  10  U.S.C. 
§  1566,  and  10  U.S.C.  §  1566a)  provides  for  a  presidential  designee,  VAOs,  IVA  offices, 
and  state  and  local  points  of  contact.  Policy  (DoDI  1000.04  [2012],  DoDD  5124.02 
[2008],  and  service-level  guidance),  elaborates  on  the  form  and  substance  of  the  inter¬ 
faces.  For  example,  each  of  the  services  maintains  its  own  voting  assistance  program, 
consistent  with  both  the  DoD-level  guidance  and  its  own,  service-specific  guidance. 
And,  the  private  sector,  largely  in  the  form  of  NGOs,  offers  a  range  of  services,  consist¬ 
ing  of  complements  to  and  substitutes  for  publicly  provided  services,  such  as  web-based 
voting  tools. 

Connectivity  and  Coordination  Vary  Across  the  System 

Not  surprisingly,  in  a  system  that  was  not  developed  explicitly  as  a  system,  the  parts  do 
not  mesh  together  seamlessly.  To  the  extent  that  there  is  a  system,  it  reflects  an  ad  hoc 
buildup  of  legal  and  policy  requirements  in  the  context  of 

•  competing  priorities 

•  limited  capital  and  human  resources 

•  turnover,  especially  among  ADM  service  members 

•  cultural  and  environmental  differences  across  services  and  venues. 

The  law  forces  some  amount  of  connectivity  in  its  implicit  call  for  relationships 
between  FVAP  and  the  states  and  localities  and  between  FVAP  and  a  host  of  federal 
entities,  including  the  services,  but  it  speaks  more  to  the  responsibilities  of  each  entity 
than  to  the  nature  of  the  relationships  among  them.  Notably  absent  from  the  MOVE 
Act  is  any  indication  of  how  the  IVA  offices  would  operate  in  the  context  of  existing 
service  providers.  Rather,  the  law  appears  to  have  taken  a  gap  in  service — pertaining 
largely  to  voter  registration — as  given  and  fillable,  independently  of  other  actors  or 
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actions  in  the  system.9  Similarly,  DoDI  1000.04,  2012,  spells  out  responsibilities  and 
procedures  for  a  set  of  federal  providers  bur  says  less  about  how  they  should  work 
together.  To  the  extent  that  either  DoDI  1000.04  or  DoDD  5124.02,  2008,  speaks 
to  relationships,  it  does  so  largely  in  the  context  of  the  chain  of  command  in  which 
a  provider  resides.  DoDI  1000.04  offers  sparing  insight  to  the  intended  relationships 
between  SVAOs,  IVAO,  and  UVAOs.  It  leaves  the  details  of  those  relationships  to  each 
of  the  services  and  to  their  respective  service-specific  guidance.10 

The  System  and  Its  Parts  Need  Knowledge,  Capability,  and  Capacity 

The  system,  consisting  of  the  institutions  that  function  within  it,  the  people  that  popu¬ 
late  the  institutions,  and  the  laws  and  policies  that  govern  each,  has  components  that 
differ  in  their  priorities,  but  the  system  and  its  parts  all  need  a  combination  of  capabili¬ 
ties  and  capacities,  rooted  in  knowledge  and  skills,  to  function  either  together  or  sepa¬ 
rately.* 11  People,  institutions,  and  the  system  require  an  understanding  of  their  rights, 
roles,  and  responsibilities  and  of  voting  processes,  tools,  and  resources.  On  that  basis, 
the  system  and  its  parts  can  be  said  to  share  an  interest  in  developing  and  maintaining 
the  capability  and  capacity  to  serve  voters,  which,  in  turn,  requires  the  accumulation 
of  knowledge  and  skills.  Admittedly,  they  share  the  interest  subject  to  resource  con¬ 
straints,  such  as  those  relating  to  funding,  staffing,  and  turnover,  and  to  other  environ¬ 
mental  considerations,  but  they  still  share  the  interest.12 

One  might  also  argue  that,  as  a  dimension  of  capacity,  the  system  and  its  parts 
require  connectivity  and  coordination.  Lesser  amounts  of  both  could  imply  greater 
amounts  of  inefficiency  (e.g.,  in  the  form  of  duplication)  and  higher  rates  of  voting  fail¬ 
ure.  For  example,  if  one  part  of  the  system  fails  to  dovetail  with  another,  as  in  the  case 
of  timely  receipt  or  delivery  of  balloting  materials,  votes  might  not  be  counted. 

The  Federal  Voting  Assistance  Program  Faces  Substantial  Functional  Limitations 

Two  factors  contribute  to  FVAP’s  functional  limitations:  First,  as  a  very  small  agency, 
tasked  with  administering  a  law  that  serves  many  voters,  it  operates  through  and 
depends  largely  on  intermediaries;  second,  it  has  little  direct  control  over  intermediaries. 


9  This  observation  draws  from  stakeholder  outreach,  including  conversations  with  congressional  staff  held 
during  phase  1,  and  from  Senator  Chuck  Schumer’s  statement  to  accompany  the  MOVE  Act  (Schumer,  2010). 

10  It  also  leaves  such  details  to  the  U.S.  Coast  Guard,  the  Department  of  State,  and  the  Public  Health  Service. 

11  Perhaps  in  recognition  of  these  needs,  law  mandates  training,  and  policy  specifies  the  terms  of  training. 

12  The  system  and  its  parts  express  or  reveal  their  needs  through  actions  and,  in  some  instances,  through  surveys 
and  evaluations,  but  that  evidence  is  mixed.  For  example,  we  found  that  attendance  at  training  sessions  varied 
greatly  but  that  many  of  those  in  attendance  strongly  preferred  in-person  training  to  other  options.  Given  the 
challenges  of  scheduling  across  units  and  differences  in  how  the  installations  issue  invitations  to  prospective 
trainees — ranging  from  minimally  distributed  suggestion  to  comprehensive  edict — weak  attendance  at  some 
locations  might  be  more  reflective  of  poor  coordination  than  of  need;  alternatively,  the  expressed  preferences  for 
in-person  training  might  reflect  the  self-selection  biases  of  those  present. 
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The  Federal  Voting  Assistance  Program  Operates  Through  and  Depends  Largely  on 
Intermediaries 

Much,  but  not  all,  of  FVAP’s  role  in  voting  assistance  is  realized  and  experienced 
through  intermediaries  who  serve  UOCAVA  voters  more  directly.  Through  the  FVAP 
website,  the  customer  call  center,13  the  ETS,  and  training,  FVAP  staff  stands  adjacent 
to  or  only  one  degree  removed  from  the  voter,  but  FVAP  often  operates  at  a  greater  dis¬ 
tance  as  an  enabler.  In  the  military  context,  FVAP  works  largely  through  the  SVAOs 
who,  in  turn,  reach  out  to  IVAOs,  who  then  make  contact  with  UVAOs,  who  then 
interact  more  directly  with  voters.  Moreover,  we  found  that  IVAOs  and  IVA  office  staff 
typically  reach  out  to  their  SVAOs  when  need  arises,  in  accordance  with  the  structure 
of  their  programs,  and  less  characteristically  make  contact  with  FVAP,  except  in  rela¬ 
tion  to  training  and  to  request  FPCAs,  FWABs,  and  other  materials.  This  pattern  of 
engagement  is  not  inconsistent  with  law  or  policy,  but  its  manifestation — and,  espe¬ 
cially,  its  implications — became  more  apparent  with  site  visits  and  training  observa¬ 
tions.  In  terms  of  the  logic  model,  FVAP  has  limited  control  over  transfer  and  little 
means  to  affect  the  use  of  outputs,  the  actions  of  customers,  or  eventual  outcomes. 

In  its  direct  service  to  UOCAVA  voters,  FVAP  extends  its  reach  through  elec¬ 
tronic  means,  such  as  the  web-based  voting  assistance  tools,  email  blasts,  and  social 
media.14  However,  even  as  a  direct  provider,  FVAP  depends  partly  on  others,  especially 
VAOs,  to  raise  awareness  and  disseminate  information  about  its  offerings.  Much  of 
what  FVAP  asks  VAOs  and  others  to  do  is  to  direct  voters  to  FVAP’s  website  and  its 
resources. 

Table  6.2  provides  further  insight  into  the  roles  of  intermediaries  in  the  services, 
consisting  of  the  SVAOs,  VAOs  (both  IVAOs  and  UVAOs),  and  IVA  offices  or  office 
managers.  In  highly  stylized  terms,  it  presents  and  compares  the  theory  of  their  roles, 
according  to  law  and  policy,  and  the  practice,  according  to  our  site  visits  and  training 
observations. 

By  and  large,  theory  and  practice  lined  up.  For  example,  our  conversations  with 
VAOs  and  IVA  office  managers  and  information  gleaned  from  the  training  sessions 
pointed  to  a  substantial  amount  of  direct  contact  between  UVAOs  and  ADM  voters, 
especially  in  the  run-up  to  presidential  elections.  But  we  found  three  areas  in  which 
theory  and  practice  did  not  line  up. 

First,  we  found  little  evidence  of  an  active  interface  between  the  IVA  offices,  per 
se,  and  ADM  or  other  UOCAVA  voters;  rather,  the  offices  were  functionally  indis¬ 
tinguishable  from  the  IVAOs,  who  often  serve  as  IVA  office  managers.15  The  quality 


13  At  the  time  of  phase  2  of  the  project,  FVAP  was  in  the  process  of  bringing  its  customer  call  center  in  house,  as 
a  voting  assistance  center,  thereby  providing  another  channel  for  serving  voters — and  others — directly. 

14  FVAP  also  serves  voters  directly  through  the  call  center,  but  in  much  smaller  volume. 

15  On  at  least  one  installation,  in-  and  out-processing,  which  are  functions  associated  with  IVA  offices  under 
10  U.S.C.  §  1566a,  were  handled  at  the  unit  level  primarily  and  reported  through  the  IVA  office  manager. 
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Table  6.2 

Roles  of  Intermediaries  in  the  Services 


Primary  Function 

Primary  Audience 

Intermediary 

Theory 

Practice 

Theory 

Practice 

SVAO 

Service-level 
program  managers, 
conduits,  and 
resources  (policy 
implementation) 

Service-level 
program  managers, 
conduits,  and 
resources  (policy 
implementation) 

IVAOs,  IVA  offices, 
or  office  staff3 

IVAOs,  IVA  offices, 
or  office  staff 

IVAO 

Installation-level 
coordinators, 
conduits,  and 
resources13 

Installation-level 
coordinators, 
conduits,  and 
resources 

UVAOs  (resident, 
tenant) 

UVAOs  (resident, 
tenant) 

UVAO 

Unit-level  rubber- 

meets-the-road 

assistance*- 

Unit-level  rubber- 

meets-the-road 

assistance 

ADM  and 
dependents 

ADM 

IVA  office  or  office 
staff,  including 
managers 

Installation-level 
rubber-meets-the- 
road  assistance 
(fixed  location)  with 
NVRA  function 

See  IVAO,  with 
less  emphasis  on 
coordination 

ADM,  dependents, 
and  others 

See  IVAO 

SOURCE:  RAND  staff  analysis,  based  on  combination  of  policy  documents  and  direct  observation. 
a  SVAOs  also  have  direct  contact  with  UVAOs,  but  we  did  not  view  UVAOs  as  their  primary  audience. 

b  IVAOs  are  available  to  assist  voters  (hence  their  inclusion  in  Figure  6.1),  but  it  is  our  understanding  that 
they  are  intended  to  play  a  more  coordinative  primary  role,  and  they  appear  to  do  so. 

c  Assistance  includes  direction  to  FVAP.gov. 


of  the  installation-based  program  appeared  to  depend  on  the  level  of  commitment, 
knowledge,  and  skills  of  the  IYAOs,  subject  to  the  amount  of  time  they  had  available 
to  perform  the  IVAO  function,  be  it  a  primary,  secondary,  or  «th-order  duty,  and  their 
access  to  the  installation-level  chain  of  command.16  It  appeared  to  be  less  dependent 
on  the  quality  of  the  physical  office  space  or  its  location.  Admittedly,  we  visited  a  small 
number  of  installations,  but  the  picture  was  consistent  across  sites  and  appeared  to  be 
unrelated  to  the  placement,  layout,  or  marketing  of  the  offices.  Even  those  IVA  offices 
located  in  high-traffic  areas  (for  example,  near  a  TRICARE  office)  with  good  signage 
saw  few  voters,  according  to  those  who  staff  the  offices. 

Second,  relating  to  the  fixity  of  IVA  offices,  we  found  that  the  location  of  an 
ADM-managed  office  tended  to  be  tied  to  the  location  of  the  appointee’s  primary 
responsibilities.  The  office  could  move  with  each  new  appointee,  depending  on  where 
that  appointee  had  been  assigned  an  office  or  cubicle  to  execute  his  or  her  primary 
duties. 


16  On  that  basis,  we  would  expect  to  see  high-functioning  programs  on  Army  installations  with  dedicated 
IVAOs.  Early  reports  from  within  the  Army  and  from  FVAP  provided  anecdotal  confirmation. 
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Third,  UVAOs  appeared  to  reach  dependents  largely  through  military  service 
members.  Insomuch  as  they  described  direct  contact  with  dependents,  it  was  usually 
through  special  events,  such  as  those  held  during  voter  emphasis  weeks.  For  exam¬ 
ple,  some  UVAOs  mentioned  engagement  with  spouses  at  post-exchange  registration 
booths. 

The  Federal  Voting  Assistance  Program  Has  Little  Direct  Control  over  Intermediaries 

Neither  law  nor  policy  gives  FVAP  much  direct  control  over  its  intermediaries,  be 
they  within  or  outside  DoD.  The  law  (e.g.,  42  U.S.C.  §  1973ff,  10  U.S.C.  §  1566, 
and  10  U.S.C.  §  1566a)  holds  FVAP  responsible  for  developing,  maintaining,  and 
providing  processes,  tools,  and  resources  for  conducting  certain  types  of  outreach  and 
for  reporting  on  effectiveness  (see  Chapter  Four).  The  last  responsibility,  reporting  on 
effectiveness,  might  be  viewed  as  a  sunshine  tool — that  is,  a  means  to  shed  light  on 
emerging  issues  not  just  within  FVAP  but  also  across  the  system.  In  those  ways,  FVAP 
can  influence  the  behavior  of  at  least  some  intermediaries,  but  the  law  holds  others, 
such  as  the  services,  IG,  DOJ,  and  states,  responsible  for  most,  if  not  all  of  implementa¬ 
tion,  oversight,  compliance,  and  enforcement. 

The  relationship  between  policy  and  control  merits  consideration,  in  part,  because 
FVAP  actively  engages  in  policymaking.  Policy,  if  treated  as  an  external  factor,  e.g., 
postdating  its  issuance  and  prior  to  its  revision,  provides  FVAP  with  little  authority 
over  the  actions  of  others.  It  gives  FVAP  an  administrative  role  in  relation  to  its  inter¬ 
mediaries,  but  it  does  not — and  cannot — place  FVAP  in  their  chains  of  command.  As 
such,  FVAP  can  serve  those  in  other  chains  as  customers  and  work  with  them  in  part¬ 
nerships,  but  it  is  not  their  supervisor.  However,  policy  is  not  a  fully  external  factor. 
FVAP  plays  a  substantial  part  in  the  development  of  policy,  particularly  through  the 
DoD  instruction  on  UOCAVA  voting.  Thus,  FVAP  can  be  said  to  establish  certain 
behavioral  parameters  and  ground  rules.17 

The  Federal  Voting  Assistance  Program  Occupies  a  Unique  Position  in  the  System 

FVAP  is  one  provider  among  many,  as  is  evident  in  Figure  6.1,  but  it  also  has  unique 
qualifications  as  a  provider  that  might  be  less  readily — or  visually — apparent.  In  addi¬ 
tion  to  being  the  only  agency  chartered  through  law,  directives,  and  instructions  with 
the  primary  purpose  of  assisting  UOCAVA  voters  and  coordinating  and  establishing 
policy  on  UOCAVA  voting,  it  has,  over  time,  become  a  comprehensive  repository  of 
information  on  the  processes,  tools,  and  resources  that  it  develops  and  maintains.  As 
a  consequence,  FVAP  might  be  the  only  public  agency  with  the  credentials,  internal 
motivation,  and  dedicated  resources  to  play  a  leadership  role  in  UOCAVA  voting  assis¬ 
tance.  Moreover,  in  sitting  at  the  nexus  of  policy  and  operations  (see  Chapters  One  and 


17  It  does  not  develop  policy  alone — it  must  obtain  input  from  others,  including  the  services — but  it  coordinates 
the  process.  In  turn,  the  services  develop  their  own  policies,  plans,  and  procedures,  consistent  with  the  DoD 
instruction. 


The  Federal  Voting  Assistance  Program's  Engagement  with  Its  Operating  Environment  79 


Two),  FVAP  has,  on  the  one  hand,  a  substantial  role  in  policymaking  and,  on  the  other 
hand,  a  sweeping  view  of  the  system,  with  the  potential  for  a  high  degree  of  environ¬ 
mental  awareness. 

The  Federal  Voting  Assistance  Program  Can  Use  Training  to  Leverage  Its  Position  in 
the  System 

In  scoping  the  voting  assistance  system,  we  found  that  FVAP  has  surprisingly  few 
opportunities  to  reach  proactively  into  the  system  on  a  regular  basis18  and  that  train¬ 
ing  stands  out  as  an  exceptional  touch  point.  Through  training,  FVAP  can  engage  its 
intermediaries,  particularly  VAOs  and  IVA  office  staff,  more  directly  and  get  closer  to 
UOCAVA  voters.  Thus,  training  appears  to  hold  value  not  just  as  an  educational  tool 
but  also  as  a  potential  source  of  ancillary  benefits  or  gains  for  FVAP  and  the  system. 
Our  conversations,  site  visits,  and  training  observations  strongly  suggested  that  FVAP 
can  use  training  to  promote  the  system’s  functionality  by  helping  to  build  networks, 
raising  FVAP’s  profile  as  a  resource,  signaling  the  importance  of  voting  assistance, 
providing  FVAP  with  access  to  information  about  operating  conditions  and  feedback 
from  VAOs  and  IVA  office  staff,  and  encouraging  connectivity  and  coordination  across 
the  system.  Training  also  represents  an  opportunity  for  FVAP  to  deliver  its  message. 

However,  whether  training  can  produce  these  ancillary  benefits  depends  partly 
on  the  modality  of  training — that  is,  whether  it  is  in  person  or  online — and  partly  on 
steps  that  FVAP  takes  to  elicit  them.  In  the  next  section,  we  summarize  our  analysis  of 
and  findings  on  the  VAO  training  program  and  address  these  issues. 


Summary  of  Training  Analysis  and  Findings 

The  law  does  not  specify  a  role  for  FVAP  in  VAO — or  IVA  office — training  (Chapter 
Four),  but  that  lack  of  specificity  need  not  imply  a  lack  of  necessity  or  efficacy.  FVAP 
serves  as  a  repository  of  knowledge  of  voting  assistance  and  tools  and,  on  that  basis 
alone,  might  be  well-positioned  to  play  a  significant  role  in  training.  The  potential 
for  ancillary  benefits,  noted  above,  suggests  another  reason  for  FVAP’s  involvement. 
However,  whether  that  involvement  means  developing  training  materials,  providing 
in-person  training,  offering  online  training,  or  undertaking  some  of  each  remains  to 
be  determined. 

This  section  summarizes  our  analysis  of  and  findings  on  FVAP’s  options  for  the 
VAO  training  program.  For  a  more  detailed  discussion  of  the  analysis  and  findings,  we 
direct  the  reader  to  Appendix  E,  which  presents  the  analysis  more  fully,  and  Appen¬ 
dix  F,  which  focuses  on  principles  of  adult  learning  and  training  evaluation. 


18 


The  call  center  enables  direct  contact  with  intermediaries  and  voters,  but  it  does  so  reactively. 
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Framing  the  Options 

To  start,  we  turned  to  law  and  policy  to  set  out  the  range  of  options.19  Given  the  lack 
of  specificity  in  the  law  and  the  potential  to  revise  the  DoD  instruction  (Chapter  Four), 
FVAP’s  options,  at  least  in  the  medium  to  long  term,  appeared  to  range  from  doing 
nothing — as  in,  leaving  the  training  to  others,  such  as  the  services — to  doing  it  all. 
And,  doing  it  could  involve  in-person,  online,  and  hybrid  delivery  modes. 

We  treated  doing  it  well  as  a  necessary  condition,  one  that  is  consistent  with  the 
stakeholders’  view  that  FVAP  should  model  the  ideal  or  be  best  in  class  and  with 
the  call  for  greater  consideration  of  effectiveness.20  To  assess  the  agency’s  potential  for 
excellence  and  as  a  basis  for  developing  recommendations  on  program  improvements, 
we  defined  doing  it  well  in  terms  of  established  principles  of  adult  learning  and  training 
evaluation.  We  used  the  principles  to  identify  a  set  of  best  practices,  which  we  mapped 
to  detailed  recommendations.21  In  considering  whether  FVAP  is  or  could  be  doing  it 
well,  we  found  that  FVAP  was  doing  some  things  better  than  others  but,  with  some 
investment  in  the  training  program  (see  below  and  Appendixes  E  and  F),  could  excel 
more  uniformly. 

We  delivered — or  transferred — the  principles,  practices,  and  recommendations 
to  FVAP  in  a  daylong  workshop  that  we  led,  consisting  of  a  general  presentation,  a 
small-group  discussion  with  FVAP  trainers,  and  a  series  of  one-on-one  meetings  with 
trainers. 

Weighing  the  Evidence 

We  approached  the  analysis  of  options  for  the  VAO  training  program  in  two  steps: 
First,  we  assessed  the  benefits,  costs,  and  risk  of  adopting  the  implied  best  practices; 
second,  assuming  the  adoption  of  best  practices,  we  compared  the  notional  benefits, 
costs,  and  risks  of  each  option,  consisting  of  in-person  and  online  training  modes. 
Figure  E.l  in  Appendix  E  describes  the  two  steps  as  elements  of  a  systematic  decision¬ 
making  process. 

To  conduct  the  analysis,  we  drew  on  the  project-management  tools  (Appendix  D); 
on  our  site  visits  and  direct  observations  of  training;  on  our  conversations  with  VAOs 


19  Although  we  use  the  term  options  in  this  discussion,  we  note  that  FVAP  cannot  choose  a  course  in  isolation — 
that  is,  without  reference  to  or  consultation  with  other  parties  in  the  system. 

20  In  this  way,  we  also  departed  from  our  approach  to  the  requirements  analysis  in  phase  1,  in  which  we  consid¬ 
ered  whether  FVAP  was  covering  a  requirement  but  not  whether  it  was  covering  it  adeptly. 

21  To  illustrate,  we  found  that  adult  learners  need  to  know  why  they  are  learning,  which,  in  turn,  implied  that 
trainers  must  spell  out  the  purpose  and  objectives  at  the  start  of  each  training  event.  We  present  the  complete  set 
of  principles,  practices,  and  detailed  recommendations  in  Appendix  F.  The  recommendations  are  consistent  with 
the  U.S.  Army  Training  and  Doctrine  Command  (TRADOC)  Pamphlet  (PAM)  525-8-2,  The  U.S.  Army  Learn¬ 
ing  Concept  for  2015  (U.S.  Department  of  the  Army,  2011).  The  pamphlet  describes  a  learner-centered  learning 
model,  focused  on  context-based,  collaborative,  problem-centered  instruction. 

In  Appendix  F,  we  also  present  a  sample  evaluation  form  that  is  consistent  with  the  learner-centered  approach 
to  participant  feedback  (D.  Kirkpatrick,  1978). 
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and  IVA  office  staff;  and  on  additional  conversations  with  FVAP  staff  during  the  day¬ 
long  workshop.  In  the  course  of  the  workshop,  we  sought  feedback  from  FYAP  staff 
and — a  testament  to  the  value  of  the  interaction — worked  with  their  comments  in  the 
analysis  of  options. 

On  balance,  the  best  practices  looked  promising.  The  benefits  of  adopting  them 
could  be  substantial  and  recurring,  e.g.,  in  terms  of  learning  and  retention  and,  hence, 
capability  and  capacity,  and  the  costs  appeared  be  modest  and  mostly  one-time,  con¬ 
sisting  of 

•  professional  development 

•  modifications  to  course  materials 

•  redeployment  of  materials. 

We  saw  few  downside  risks  to  adopting  best  practices.  A  trainer  might  leave  FYAP 
and  take  his  or  her  human  capital  along,  but,  if  acting  as  a  mentor  to  other  staff  or  a 
replacement,  at  least  some  capital  would  be  transferred  before  departure. 

Taking  the  adoption  of  best  practices  as  given,  we  then  asked,  “How  do  the  ben¬ 
efits,  costs,  and  risks  of  in-person  and  online  training  modes  compare?”  Tables  E.2, 
E.3,  and  E.4  in  Appendix  E  set  out  the  results  of  our  analysis  of  the  options. 

We  found  the  following: 

•  The  benefits  of  in-person  training,  much  more  so  than  of  online  training,  can 
extend  beyond  those  of  the  initial  learning  experience  and  spread  across  the 
system  through  networking,  signaling,  and  other  methods,  but  online  training 
can  be  accessed  globally,  24/7. 

•  The  costs  of  in-person  training  are  likely  to  be  greater  than  those  of  online  train¬ 
ing,  especially  if  IT  is  already  in  place  and  available  on  installations. 

•  Each  mode  presents  risks  to  the  individual  learning  experience  insomuch  as  it 
might  fail  to  meet  the  learning  needs  of  people  who  prefer  the  other  mode,  pres¬ 
ent  challenges  of  availability,  or  entail  environmental  distractions,  but  some  insti¬ 
tutional  and  systemic  risks  pertaining  to  coordination  and  turnout  are  specific  to 
in-person  training. 

The  analysis  also  supported  continuance  of  a  mixed  strategy  of  in-person  and 
online  training  and  implementation  of  programmatic  improvements.22  The  combina¬ 
tion  of  benefits,  costs,  and  risks  pointed  to  the  continuance  of  both  modes  of  training, 


22  In  a  manner  that  is  consistent  with  our  recommendation  for  a  mixed  strategy,  Army  training  doctrine  describes 
the  need  for  blended  learning.  Blended  learning  is  the  combination  of  online  or  technology-delivered  instruc¬ 
tion  and  face-to-face  instruction:  “It  blends  the  efficiencies  and  effectiveness  of  self-paced,  technology-delivered 
instruction  with  the  expert  guidance  of  a  facilitator,  and  can  include  the  added  social  benefit  of  peer-to-peer 
interactions”  (U.S.  Department  of  the  Army,  2011,  p.  19).  See  U.S.  Department  of  the  Army,  2011,  for  more 
information. 
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responsive  to  differences  in  learning  styles,  differences  in  service  cultures  and  settings, 
the  potential  for  ancillary  gains,  and  the  risks  of  competing  demands  on  personnel  and 
of  staff  turnover,  especially  among  ADM  staff.  Moreover,  it  suggested  several  avenues 
of  improvement,  including  the  possibilities  of  rebalancing  through  streamlining,  tar¬ 
geting,  and  tailoring  and  working  with  local  power  brokers  to  mitigate  the  risks  of 
coordination  failures.  We  also  identified  opportunities  to  build  ancillary  gains  into  the 
training  program.  Lastly,  the  call  to  “do  it  well”  and  the  initial  analysis  of  the  costs, 
benefits,  and  risks  of  adopting  best  practices  suggested  both  the  adoption  of  best  prac¬ 
tices  and  a  need  for  professional  development. 

We  concluded  our  analysis  with  a  set  of  specific  suggestions  for  strengthening  the 
VAO  training  program.  In  broad  terms,  we  suggested  that  FVAP  “maintain  [its]  train¬ 
ing  profile,  but  do  it  better  and  smarter”  by 

•  adopting  best  practices 

•  streamlining  in-person  engagement 

•  reducing  risks  of  coordination  failures  with  training  venues 

•  building  the  attainment  of  ancillary  gains  into  the  training  program.23 

Adopt  Best  Practices 

We  offered  detailed  recommendations  on  learning  objectives,  interactivity,  knowledge 
transfer,  and  evaluations  (Appendix  F)  and  suggested  professional  development  to  fill 
gaps  in  capability  and  capacity.24  We  recognized  that  making  in-person  training  more 
interactive  would  require  a  paradigm  shift  and  that,  to  succeed,  trainers  would  need  to 
be  able  to  conduct  training  responsively  but  stay  on  track.  Our  suggestions  tied  back 
to  the  overarching  themes  of  phase  1  regarding  two-way  communication,  e.g.,  do  not 
just  broadcast  to  stakeholders  but  also  receive,  process,  and  respond  to  the  information 
and  feedback  they  provide. 

Streamline  In-Person  Training 

We  recommended  that  FVAP  offer  regional  and  targeted  training  sessions,  develop 
audience-specific  training  modules,  and  cultivate  relationships  with  alternative  pro¬ 
viders.  With  more  regional  and  better-targeted  training  sessions,  FVAP  could  visit 
fewer  installations  overall.  For  example,  it  might  identify  domestic  and  overseas  hot 
spots,  large  markets,  and  other  high-impact  markets  and  use  training  as  an  opportu¬ 
nity  both  to  improve  faltering  relationships  and  to  maintain  healthy  and  productive 
relationships.  In  visiting  fewer  installations,  but  with  greater  purpose,  FVAP  might 
reduce  its  operating  costs — and  those  of  the  services — and  still  increase  the  benefits  of 


23  For  additional  details  on  each  of  these  points,  see  the  discussion  in  Appendix  E. 

24  We  also  pointed  FVAP  to  professional  associations  and  websites  that  offer  relevant  training  programs  and 
provide  downloadable  instructional  materials. 


The  Federal  Voting  Assistance  Program's  Engagement  with  Its  Operating  Environment  83 


the  program.  We  also  suggested  developing  train-the-trainer  and  IVAO-specific  train¬ 
ing  modules  for  online  and  in-person  delivery.  Lastly,  we  recommended  cultivating 
relationships  with  installation-based  spousal  and  dependent  organizations  and  other 
NGOs,  including  those  overseas,  to  lessen  the  burden  on  FVAP  trainers  and  training. 

Reduce  Risks  of  Coordination  Failures  with  Training  Venues 

To  address  coordination  failures,  such  as  inadequate  site  preparation  and  weak  turn¬ 
out,  we  suggested  that  FVAP  ramp  up  its  efforts  to  identify  and  work  with  power  bro¬ 
kers  on  and  off  installations.  Across  venues,  we  observed  the  positive  effects  of  issuing 
training  invitations  from  appropriate  authority  figures,  using  institutionally  appropri¬ 
ate  vocabulary.  In  these  cases,  attendance  was  high;  in  other  cases,  it  was  not.  Our  site 
visits  were  few,  but  the  observation  was  striking.  As  a  related  matter,  we  also  empha¬ 
sized  the  importance  of  working  within  institutional  norms  and  at  appropriate  institu¬ 
tional  levels.  In  some  instances,  it  might  be  necessary  for  the  director  to  reach  out  to 
his  or  her  counterpart  or  bureaucratic-hierarchical  equivalent;  in  others,  it  should  be 
sufficient  to  operate  at  the  staff  level. 

Build  the  Attainment  of  Ancillary  Gains  into  the  Training  Program 

In  formulating  this  recommendation,  we  took  as  a  given  that  ancillary  benefits  would 
not  accrue  automatically  and  would  require  some  additional  thought,  planning,  and 
action  on  the  part  of  FVAP.  We  addressed  four  areas — namely,  networking,  signaling, 
resource  exposure,  and  information  collection: 

•  Use  training  events  to  build  stronger  networks  by  encouraging  interaction  among 
trainees,  both  during  and  after  training,  encouraging  repeat  performers  at  all 
levels  (IVAOs,  VAO  office  staff,  and  IVAOs),  and  emphasizing  the  value  of  using 
continuity  folders  to  pass  down  lessons  learned. 

•  Use  physical  presence  to  signal  the  importance  of  voting  assistance  through  sys¬ 
tematic  implementation  of  meet-and-greets  and  site  visits,  both  on  and  off  base. 

•  Use  training  events — in  person  and  online — to  increase  trainees’  direct  exposure 
to  FVAP  resources.  This  could  be  accomplished  through  simulated  links  in  in- 
person  training  and  through  direct  links  embedded  in  exercises  in  online  train¬ 
ing. 

•  Gather  and  obtain  information  intentionally,  and  develop  mechanisms  to  capture 
and  use  new  knowledge  productively.  For  example,  FVAP  could  collaborate  with 
the  services  to  develop  mechanisms  to  capture  lessons  from  continuity  folders  and 
share  them  intergenerationally,  across  installations,  and  even  across  services. 

Lastly,  we  suggested  that  FVAP  view  training  as  an  opportunity  to  spread  its  mes¬ 
sage  consistently  across  venues.  To  do  so,  it  would  need  to  reevaluate  its  in-person  and 
online  training  materials  as  outreach  and  take  steps  to  ensure  that  the  training  materi¬ 
als  convey  the  same  message  as  other  outreach  materials. 
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Further  Development  of  the  Logic  Model 

In  phase  2,  we  also  worked  with  FVAP  on  the  development  and  refinement  of  the 
robust  logic  model.  That  work  drew  heavily  from  the  recommendations  and  guidance 
outlined  in  Chapter  Five,  e.g.,  by  incorporating  ideas  for  agency-wide  integration.  We 
introduced  a  draft  version  at  the  close  of  our  discussions  with  FVAP  staff  on  the  find¬ 
ings  of  phase  1 .  We  made  only  a  few  changes  to  the  model  in  the  months  that  followed, 
so  we  present  the  final,  robust  version  here  (Figure  6.3),  as  groundwork  for  the  discus¬ 
sion  of  change  in  Chapter  Seven. 

Perhaps  the  most-noteworthy  features  of  the  robust  logic  model  are  the  reformu¬ 
lation  of  voter  assistance  to  include  all  forms  of  assistance,  be  they  direct  to  voters  or 
through  the  states,  and  the  inclusion  of  the  technology  that  undergirds  that  assistance. 
Those  features  made  it  possible  to  eliminate  much  of  the  repetition  that  we  observed  in 
the  benchmark  model,  to  streamline  the  agency’s  activities  and  outputs,  and  to  stream¬ 
line  our  presentation  of  intermediate  customers.  In  specifying  FVAP  as  “one  FVAP” 
with  regard  to  voter  assistance,  it  was  no  longer  necessary  to  assign  customers  to  par¬ 
ticular  activity  streams;  rather,  the  model  recognizes  the  agency’s  customers  as  relating 
to  the  agency,  as  a  whole. 

Another  difference  in  the  robust  model  is  its  comparative  generality.  Whereas 
the  benchmark  model  was  full  of  specific  examples  of  guidance,  media,  tools,  data, 
statistics,  and  special  projects,  this  version  has  few  or  none.  It  was  necessary  to  include 
those  details  in  the  benchmark  model  so  that  FVAP  and  we  could  see  the  agency  in 
its  entirety  and  substitute  for  a  shared  vocabulary  for  describing  its  operations.  FVAP 
leadership  and  staff  needed  to  be  able  to  locate  themselves  in  the  model;  without  the 
examples,  we  could  not  be  certain  that  everyone  was  using  words,  such  as  guidance , 
media ,  or  tools,  in  the  same  way.  With  a  fuller  understanding  of  the  agency  and  a 
shared  vocabulary,  it  was  not  necessary  to  include  as  many  details  in  this  version.  Nev¬ 
ertheless,  the  rearticulated  model  covers  all  the  core  requirements  set  out  in  Table  4.1 
in  Chapter  Four.  If  FVAP  were  to  follow  this  model,  it  would  have  some  assurance  that 
it  was  undertaking  the  activities  necessary  to  meet  those  requirements. 

In  summary,  we  claim  robustness  for  the  model  in  at  least  three  regards:  First, 
it  draws  together  all  forms  of  voter  assistance  in  a  single,  coherent  activity  stream; 
second,  with  fewer  specifics,  it  should  be  less  susceptible  to  the  ordinary — and  even 
extraordinary — ebb  and  flow  of  activities;  third,  it  covers  FVAP’s  core  requirements. 
However,  its  robustness  will  depend  partly  on  FVAP’s  commitment  to  a  common 
understanding  of  its  mission. 
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Respond  to  information  &  service  requests 


Training  materials,  guidance,  notifications, 
alerts,  newsletters,  other  on-line  content, 
including  descriptive  material,  state  contact 
information  &  other  databases,  &  forms/tools/ 
systems;  Workshops  &  trained/informed 
intermediaries  &  voters;  Waiver 
determinations 


Data,  statistics,  graphics  &  analysis  to 
support  development  &  dissemination  of 
products  &  services,  management  & 
reporting;  legal  &  policy  analyses;  Other 
findings  relevant  to  legal  &  policy 
requirements  &  emerging  needs 


Strategy,  budgets,  plans  &  policy 
recommendations;  Program  evaluations  & 
performance  reviews;  Contracts,  logistics 
&  staff  qualifications;  PR/media  training  & 
messaging  materials,  guidance  &  tools; 
Clear/consistent  message,  embodied  in  all 
communications,  products  &  services 


Reports,  presentations,  Congressional  testimony/ 
letters,  notifications  &  other  media;  Strong 
relationships  with  stakeholders,  implying  access, 
engagement  &  collaboration;  Direct  responses  to 
information  &  service  requests 


Partnerships  with  DoD  agencies  (e.g.,  DHRA,  Services/SVAOs,  OSD/P&R,  PA  &  LA,  DMDC,  DTIC,  MPSA,  WHS,  Forms), 
other  federal  agencies  (e.g.,  DOS,  DOJ,  NIST,  EAC,  USPS,  OMB,  OPM,  GSA),  election  officials,  NGOs,  contractors 


Monitor  needs,  outcomes  &  impact 


Transfer: 

Via  website  &  \ 

media  channels;  \ 
Bulk  email,  ETS, 
Voting  Assistance 
Center;  Services, 
DOS,  election 
official  &  NGO 
pass  through; 
Conferences, 
hearings,  site 
visits  &  other  I 
person-to-  j 
person  j 

contact  I 


a  o 


Changes  in  knowledge,  attitude,  behavior,  resulting  ii 
reductions  in  barriers  to  voting 


Intermediate 

outputs: 

Reports, 
presentations, 
testimony  & 
media  releases; 
Federal  &  state 
legislation, 
regulations, 
policy  &  plans; 
Standards  & 
guidance; 
Commentary; 
Technologies; 
Academic 
insights, 
research 
agendas  & 
findings 


Intermediate 

outcomes: 

Awareness  of 
requirements/ 
needs  & 
resources  to 
meet  them; 

Use  of 
resources; 
Ability  of 
Services/ 
DOS,  states/ 
localities  & 
NGOs  to 
serve  voters* 


Improvements 
in  Services/ 
DOS,  states/ 
localities  & 
NGOs’ 
delivery  of 
services  to 
voters, 
relating  to 
registration, 
balloting  & 
other  voting 
processes 


Intermediate 

outcomes: 


Awareness  of 
right  to  vote  & 
resources  to 
do  so; 

Use  of 
resources; 
Ability  of 
voters*  to 
participate 
effectively  in 
voting 


Increases  in 
engagement  in 
voting 


Reductions  in 
incidence  of 
failed  voting 


*UOCAVA 

- ►  =  feedback  &  interactions 


External  factors:  federal  &  state  policy  &  legislation;  funding  levels  (FVAP,  stakeholders);  postal  delivery;  voting  technology/security  &  requirements;  IG,  GAO  &  media  reports;  DOJ  actions;  Congressional,  NGO  &  other  stakeholder  interventions;  demographics 

i  t  t  i  i  i  i  t  t  t  i  i  i  i  t  f 


SOURCE:  RAND  staff  analysis  with  input  from  FVAP  leadership  and  staff. 

RAND  RR882-6.3 
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Concluding  Remarks  on  the  Federal  Voting  Assistance  Program's 
Engagement  with  Its  Operating  Environment 

In  phase  2  of  the  project,  we  fleshed  out  our  understanding  of  the  voting  assistance 
system  and  FVAP’s  roles,  responsibilities,  and  relationships  in  that  system.  Our  analy¬ 
sis  of  FVAP’s  strategy  and  operations  in  phase  1  established  the  need  for — and  laid  the 
groundwork  for — a  better  understanding  of  FVAP’s  engagement  with  its  operating 
environment. 

The  phase  2  analysis  largely  confirmed  the  gaps  and  disconnects  that  we  uncov¬ 
ered  in  phase  1  and  enabled  us  to  better  understand  both  the  rules  at  play  and  the 
ground  truth  and  their  implications  for  the  contours  and  conduct  of  voting  assistance. 
Perhaps  unexpectedly,  we  identified  training  as  an  opportunity  for  FVAP  to  engage 
more  effectively  with  the  system  and  its  parts.  We  found  that  training  can  be  used 
to  meet  basic  needs  for  capability  and  capacity,  both  by  imparting  knowledge  and 
by  improving  coordination  and  connectivity  with  the  system,  and  we  offered  FVAP 
detailed  recommendations  for  making  the  most  of  the  opportunity. 


CHAPTER  SEVEN 


Evidence  of  Change 


In  this  chapter,  we  shift  focus  and  recount  the  change  that  has  occurred  in  FVAP, 
both  in  response  to  our  analysis  and  independent  of  that  analysis.  Although  some 
change  began  before  our  issuance  of  the  preliminary  recommendations  and  guidance 
in  Chapter  Five,  we  use  the  content  of  Chapter  Five  to  gauge  progress  and  to  organize 
our  discussion.1 

We  explore  evidence  of  change  in  relation  to  our  recommendations  on  mission 
and  organization,  stakeholder  relationships,  and  effectiveness  (Table  7.1).  In  document¬ 
ing  change — and  steps  toward  change — we  have  not  taken  the  agency’s  word  that  it 
has  changed  or  is  changing,  nor  have  we  relied  on  our  own  professional  judgment  or 
conjecture;  instead,  we  have  sought  explicit  examples  of  differences  in  methods  and 
practice.  In  some  instances,  we  have  found  evidence  of  outright  change,  e.g.,  in  the 
2014  redesign  of  the  website2  or  through  customer  feedback;  in  other  instances,  we 
have  found  evidence  of  progress,  e.g.,  in  the  form  of  professional  development  that  can 
enable  or  facilitate  change.  Occasionally,  we  have  also  found  evidence  of  either  weak  or 
inconsistent  execution,  suggesting  that,  although  the  agency  is  making  progress,  it  still 
has  work  to  do  in  charting  the  road  ahead. 

As  is  apparent  in  Table  7.1,  much  of  the  evidence  of  change  maps  to  more  than 
one  issue  area,  suggesting  the  potential  to  address  some  issues  through  mutually  rein¬ 
forcing  measures. 

All  told,  we  found  evidence  of  change  in  the  following: 

•  the  reorientation  of  FVAP’s  mission  and  purpose 

•  the  redesign  of  the  website  and  its  content,  including  outreach  and  training  mate¬ 
rials 

•  the  reorganization  of  the  agency 


1  We  do  not  readdress  the  recommendations  presented  in  Chapter  Six  for  the  VAO  training  program  in  this 
chapter  but  return  to  them  in  the  next  chapter,  in  which  we  present  our  final  recommendations  and  guidance.  At 
the  time  of  this  writing,  FVAP  had  not  yet  had  time  to  implement  most  of  the  suggested  changes — some  or  many 
of  which  cannot  be  addressed  until  the  commencement  of  planning  for  the  next  (2016)  training  cycle. 

2  FVAP  also  redesigned  its  website  in  2008. 
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Table  7.1 

Evidence  of  Change  in  Relation  to  Preliminary  Recommendations  and  Guidance 


General 

Recommendation 

Area  of  Emphasis 

Evidence  of  Change 

Become  one  FVAP 

Come  to  terms  with 
the  mission 

•  Reorientation  of  the  mission  and  purpose 

•  Redesign  of  the  FVAP  website  and  its  content 
and  outreach  and  training  materials 

•  Reorganization  of  the  agency 

•  Reconfiguration  of  the  call  center  as  an  in- 
house  voting  assistance  center 

•  New  forms  of  engagement  with  states,  poten¬ 
tially  including  ongoing  work  with  CSG 

•  Reassessment  of  DoDI  1000.04 

Integrate  and  shore 
up  operations 

•  Reorganization  of  the  agency 

•  Reconfiguration  of  the  call  center  as  an  in- 
house  voting  assistance  center  and  related 
cross-training  of  FVAP  staff 

•  Enrollment  in  professional  development  pro¬ 
grams,  including  training  in  FIR  and  organiza¬ 
tional  management 

Sharpen  and  clarify 
the  message 

•  Reorientation  of  the  mission  and  purpose 

•  Redesign  of  the  FVAP  website  and  its  content 
and  outreach  and  training  materials 

Build  trust  and 

strengthen 

relationships 

Work  with  partners 
and  serve  customers 

•  Redesign  of  the  FVAP  website  and  its  content 
and  outreach  and  training  materials 

•  Reconfiguration  of  the  call  center  as  an  in- 
house  voting  assistance  center 

•  New  forms  of  engagement  with  states,  poten¬ 
tially  including  ongoing  work  with  CSG 

•  Outreach  to  DoDEA  schools 

•  NASS  resolution  on  voting  information 

•  Continuation  of  OMB  process  for  form  (FPCA 
and  FWAB)  revisions 

Communicate  better 
and  more  regularly 

•  Redesign  of  the  FVAP  website  and  its  content 
and  outreach  and  training  materials 

•  Customer  feedback  on  voting  assistance  center 
operations 

•  New  forms  of  engagement  with  states,  poten¬ 
tially  including  ongoing  work  with  CSG 

•  Enrollment  in  professional  development  pro¬ 
grams,  including  training  in  customer  service 
skills  and  techniques 

Operate  as  openly  as 
possible 

•  New  forms  of  engagement  with  states,  poten¬ 
tially  including  ongoing  work  with  CSG 

•  Continuation  of  OMB  process  for  form  (FPCA 
and  FWAB)  revisions 

•  Development  of  research  briefs 
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Table  7.1 — Continued 


General 

Recommendation 

Area  of  Emphasis 

Evidence  of  Change 

Embrace  a  culture 

Engage  routinely  in 

•  Use  of  findings  from  the  2012  postelection 

and  principles  of 

BCA  or  employ  other 

report  to  Congress  (FVAP,  2013b) 

effectiveness 

analytical  methods 

•  Development  of  a  dashboard3 

•  Enrollment  in  professional  development  pro¬ 
grams,  including  training  in  organizational  and 
project  management,  strategic  planning  and 
tactical  execution,  performance  measurement, 
and  problem-solving  and  data  analysis 

•  Staff  interest  in  use  of  project-management 
tools 

SOURCES:  RAND  staff  analysis;  see  Chapter  Five  of  this  report  for  a  complete  discussion  of  the  general 
recommendations  and  areas  of  emphasis. 

NOTE:  DoDEA  =  DoD  Education  Activity. 

a  The  availability  of  metrics — and  other  information — from  the  voting  assistance  center  suggests 
another  potential  avenue  of  change. 


•  the  reconfiguration  of  the  call  center  as  an  in-house  voting  assistance  center 

•  customer  feedback  on  voting  assistance  center  operations 

•  new  forms  of  engagement  with  states,  potentially  including  work  with  CSG 

•  the  reassessment  of  DoDI  1000.04 

•  enrollment  in  professional  development  programs 

•  outreach  through  DoDEA  schools,  in  coordination  with  DoDEA 

•  NASS  resolution  on  voting  information 

•  the  continuation  of  the  OMB  process  for  form  (FPCA  and  FWAB)  revisions 

•  the  development  of  research  briefs 

•  the  use  of  findings  from  FVAP’s  2012  postelection  report  to  Congress  (FVAP, 
2013b) 

•  the  development  of  a  dashboard  in  a  related  RAND  project 

•  staff  interest  in  the  use  of  project-management  tools. 

To  enable  the  reader  to  gauge  FVAP’s  progress  implementing  the  preliminary 
recommendations  set  out  in  Chapter  Five,  we  discuss  the  evidence  in  the  context  of 
the  issues  raised  in  that  chapter — namely,  becoming  one  FVAP,  building  trust  and 
strengthening  relationships,  and  embracing  a  culture  and  principles  of  effectiveness. 


Become  One  FVAP 

First,  FVAP  appears  to  be  coming  to  terms  with  its  mission.  The  agency  is  focusing  on 
more-direct,  hands-on  assistance  and  has  reformulated  its  statement  of  purpose,  reori¬ 
ented  its  organization  and  operations,  redesigned  its  website,  and  revised  its  outreach 
and  training  materials  around  such  assistance. 
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In  Chapter  Five,  we  asserted  that  merely  agreeing  to  a  mission  statement  would 
not  be  enough.  For  FVAP,  coming  to  terms  with  its  mission  would  entail  establishing 
a  set  of  primary  functions,  consisting  of  core,  supporting,  and  institutional  activities, 
which,  in  turn,  should  derive  from  an  evidence-based  analysis  of  requirements,  needs, 
and  actual  or  potential  effectiveness.  In  this  section,  we  present  evidence  of  progress  in 
each  arena.  FVAP  has  not  done  all  that  it  needs  to  do  in  any  one  arena,  but  it  is  moving 
in  the  right  direction  as  a  coherent  whole. 

FVAP’s  newly  revised  statement  of  purpose  reflects — and  announces — the  agen¬ 
cy’s  intent  to  focus  on  service  to  UOCAVA  voters.  A  comparison  of  the  old  and  new 
statements  speaks  to  the  distance  FVAP  has  traveled  in  honing  its  mission,  identifying 
means  to  support  it,  and  carrying  it  out.  Figures  7.1  and  7.2,  which  present  screenshots 
of  the  relevant  before  and  after  web  pages,  enable  a  direct  comparison.  The  old  state¬ 
ment  is  not  as  much  a  statement  as  an  amalgam  of  perspectives  on  the  meaning  of 
assistance. 

The  reasons  for  FVAP’s  existence  are  stated  similarly  in  both  screenshots,  but, 
in  the  more  recent  image,  those  reasons  are  not  offered  as  a  mission  statement  but  as 
further  elaboration  of  a  tighter  statement  of  purpose.  FVAP  has  framed  its  purpose 
succinctly  in  terms  of  awareness,  tools,  and  resources.  We  also  note  that  the  statement 
is  no  longer  buried  in  an  “About  FVAP”  tab,  accessible  only  via  a  link  located  in  small 
type  at  the  bottom-right  corner  of  the  home  page,  but  is  now  posted  under  “General 
Information”  on  the  main  banner  of  the  home  page  (see  Figure  7.3). 

And,  in  a  manner  that  is  more  consistent  with  its  core  outreach  requirements, 
FVAP  has  developed  an  “Outreach  Materials”  page  (Figure  7.4),  on  which  it  posts  the 
full  range  of  its  outreach  materials.  Previously,  those  materials  were  reachable  through 
VAO-oriented  and  other  tabs,  but  not  at  a  consolidated  one-stop-shopping  location, 
labeled  “Outreach  Materials.”  On  this  page,  the  agency  refers  to  its  posters  as  “infor¬ 
mational”  not  “motivational,”  possibly  reducing  some  confusion  about  FVAP’s  role  in 
awareness  and  motivation.  However,  FVAP  still  refers  to  them  as  “motivational”  post¬ 
ers  on  the  VAO  “Materials”  page.3  In  addition,  FVAP  offers  widgets  for  bloggers  and 
information  on  tool  kits. 

Indicative  of  a  sharper  and  clearer  message,  the  new  statement  of  purpose  features 
prominently  on  most,  if  not  all,  of  FVAP’s  outreach — and  training — materials.4 

FVAP  now  has  a  clearly  articulated  mission.  In  a  manner  that  is  consistent  with 
our  recommendation  to  appeal  to  evidence  in  decisionmaking,  FVAP  considered  find¬ 
ings  from  phase  1  of  this  project  and  from  its  own  analyses  in  the  articulation  of  the 
mission.  For  example,  it  considered  the  findings  from  our  engagement  with  stake¬ 
holders,  which  strongly  suggested  that  FVAP  is  doing  what  it  does  best  when  it  offers 


3  This  depiction  was  accurate  as  of  September  2014,  but,  at  the  time  of  publication  of  this  report,  FVAP  had 
amended  the  text  of  the  web  page  to  replace  the  word  motivational  with  informational. 

FVAP  has  not  yet  built  this  statement  into  all  of  its  training  modules. 
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Figure  7.1 

Federal  Voting  Assistance  Program  Mission,  Vision,  and  Core  Values,  Circa  2012-2013 
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About  FVAP 

Director  of  FVAP 

Vision,  Mission,  and  Core 
Values 

FVAP  Strategic  Plan 


Home  >  Mission,  Vision,  Core  Values 


Mission,  Vision  and  Core  Values 


Vision 

Military  and  overseas  voters  are  able  to  cast  a  valid  ballot,  which  is  counted  in 

the  election,  from  anywhere  in  the  world,  as  easily  as  if  they  were  at  a  polling 

place. 

Mission 

The  Federal  Voting  Assistance  Program  exists  to: 

•  Assist  uniformed  services  and  overseas  voters  exercise  their  right  to  vote  so 
that  they  have  an  equal  opportunity  with  the  general  population  to  have  their 
vote  counted. 

•  Assist  the  States  in  complying  with  relevant  federal  laws,  and  advise  them  on 
ways  to  best  comply. 

•  Advocate  on  behalf  of  the  uniformed  services  and  overseas  voters, 
identifying  impediments  to  their  ability  to  exercise  their  right  to  vote,  and 
proposing  methods  to  overcome  those  impediments. 

Core  Values 

The  Federal  Voting  Assistance  Program  operates  under  the  following  core  values: 

•  Customer-focused:  Successfully  engage  and  satisfy  FVAP's  primary 
customers:  military  and  overseas  voters  as  well  as  election  officials. 

•  Proactive:  Individual  staff  members  anticipate  future  problems,  needs  or 
changes  within  the  Department,  the  Program,  the  Services,  or  the  election 
community,  and  quickly  take  effective  action  to  prevent,  or,  if  necessary, 
resolve  those  issues. 

•  Data-driven:  Manage  programs  through  the  rigorous,  detailed  statistical 
analysis  of  the  voting  population  and  through  results-based  metrics  and 
assessments  of  voting  assistance  programs. 

•  Transparent:  Make  all  data,  methodologies,  and  performance  evaluations 
readily  available  to  the  public. 

•  Efficient:  Pursue  programs  that  stimulate  the  greatest  improvements  in 
voter  success  rates  in  the  most  expeditious  manner  and  with  the  least 
expense. 
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www.usa.gov  |  www.defenselink.mil 
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SOURCE:  Archival  snapshot  of  the  FVAP  website  (www.fvap.gov/global/mission),  retrieved  from  the 
Internet  Archive  Wayback  Machine  and  available  as  of  August  13,  2015. 
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direct  assistance  and  responds  to  the  expressed  needs  of  its  partners  and  customers.  It 
also  considered  the  statistical  analysis  of  services  and  outcomes  in  the  2012  postelec¬ 
tion  report  to  Congress  (FVAP,  2013b).  In  that  report  (FVAP,  2013b,  p.  19),  FVAP 
found  a  significant  relationship  between  interactions  with  FVAP’s  voting  assistance 
resources  and  voting  success,  defined  in  terms  of  the  likelihood  of  voting  and  return¬ 
ing  an  absentee  ballot. 

Beyond  our  direct  experience  with  the  agency  and  its  staff,  we  are  less  able  to 
provide  concrete  evidence  that  the  mission  is  shared  and  commonly  understood  within 
FVAP;  however,  in  the  context  of  that  experience,  we  have  heard  staff  describing  the 


Figure  7.2 

Federal  Voting  Assistance  Program  Statement  of  Purpose,  Circa  2014 


FVAP  Purpose 

FVAP  works  to  ensure  Service  members,  their  eligible  family  members  and  overseas  citizens  are  aware  of  their  right  to  vote  and  have  the  tools  and 
resources  to  successfully  do  so  -  from  anywhere  in  the  world 

The  Federal  Voting  Assistance  Program  exists  to 

•  Assist  uniformed  sen/ices  and  overseas  voters  exercise  their  right  to  vote  so  that  they  have  an  equal  opportunity  with  the  general  population  to 
have  their  vote  counted. 

•  Assist  the  States  in  complying  with  relevant  Federal  laws  by  providing  current  information. 

•  Advocate  on  behalf  of  the  uniformed  services  and  overseas  voters,  identifying  impediments  to  their  ability  to  exercise  their  right  to  vote,  and 
proposing  methods  to  overcome  those  impediments. 


SOURCE:  FVAP  website  image  captured  from  the  FVAP  website  on  August  27,  2014. 
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Figure  7.3 

Federal  Voting  Assistance  Program  Website  Navigation  to  Statement  of  Purpose,  Circa  2014 
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The  Federal  Voting  Assistance  Program  (FVAP)  works  to  ensure  Service  mer 
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Members  of  the  Uniformed  Services  (Army,  Navy,  Marine  Corps,  Air  F&- 


Contact 


>  Public  Health  Service  Commissioned 


Corps,  and  National  Oceanic  and  Atmospheric  Administration  Commissioned  Corps) 

•  Members  of  the  Merchant  Marines 

•  Eligible  family  members  of  the  above 

•  U  S.  citizens  residing  outside  the  U  S 

FVAP  assists  voters  through  partnerships  with  the  Military  Services,  Department  of  State,  Department  of  Justice  and  election  officials  from  50  states, 


SOURCE:  FVAP  website  image  captured  from  the  FVAP  website  on  August  27,  2014. 
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Figure  7.4 

Federal  Voting  Assistance  Program  Website  Access  to  Outreach  Materials,  Circa 
2014 


SOURCE:  FVAP  website  image  captured  from  the  FVAP  website  on  August  27,  2014. 
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agency  and  their  roles  in  ways  that  are  consistent  with  the  revised  mission  statement, 
and  we  have  observed  staff  demonstrating  new  enthusiasm  for  their  efforts. 

Indicative  of  efforts  to  come  to  terms  with  the  mission  and  integrate  and  shore  up 
its  operations,  FVAP  has  both  reorganized  its  operations  (see  Figure  7.5)  and  altered 
the  composition  of  those  operations. 

The  final,  robust  logic  model,  presented  in  Figure  6.3,  provided  a  blueprint  for 
FVAP’s  reorganization.  FVAP  has  consolidated  assistance  to  voters,  states,  and  locali¬ 
ties  and  the  technology  and  research  that  support  it,  in  a  single  line  of  operations  or 
activity  stream,  referred  to  as  the  Voting  Assistance  Division,  with  a  single  supervisor, 
a  set  of  three  SME  teams,  and  other  SMEs  with  responsibilities  for  particular,  crosscut¬ 
ting  products  and  services.  A  Communication  and  Outreach  Division,  which  includes 
a  supervisor  and  two  public-affairs  specialists,  is  managing  interactions  with  the  Con¬ 
gress  and  overseas  NGOs  and  revisions  to  DoDI  1000.04.  (The  Communication  and 
Outreach  Division  does  not  appear  to  be  managing  direct  communications  with  the 
services  or  election  officials,  which  appear  to  be  covered  under  the  Voting  Assistance 
Division.  This  division  of  labor  implies  an  ongoing  need  for  substantial — and  better — 
coordination  across  activity  streams.5)  A  Mission  Support  Division,  which  the  director 
supervises  from  the  front  office,  will  oversee  FIR,  including  contracting  and  profes¬ 
sional  development,  which  we  discuss  in  more  detail  later  in  this  chapter. 

In  the  new  chart,  there  are  three  leadership  positions,  consisting  of  the  director 
and  two  supervisory  program  analysts.  Thus,  FVAP  has  shed  two  leadership  positions, 
two  advisory  positions,  and  all  of  its  term  positions.  Regarding  the  term  positions,  it 
has  converted  some,  such  as  the  executive  assistant’s  position,  to  permanent  FTEs  and 
eliminated  others.  Moreover,  a  majority  of  the  agency’s  staff  now  serves  in  the  Voting 
Assistance  Division. 

In  addition,  FVAP  has  brought  its  call  center  in  house  as  a  voting  assistance 
center,  which  handles  calls,  emails,  live  chats,  and  faxes  from  VAOs,  voters,  election 
officials,  and  other  FVAP  stakeholders.6  Previously,  a  third-party  contractor  staffed  the 
center;  now,  FVAP  is  staffing  the  center  with  a  combination  of  core  team  members  and 
other  voter  assistance  and  communication  staff,  on  a  rotational  basis.  In  bringing  the 
center  in  house  and  providing  the  professional  development — and  cross-training — to 
make  it  possible,  FVAP  has  reduced  the  distance  between  it  and  its  customers,  both 
intermediaries  and  voters — a  central  focus  of  Chapter  Six — and,  by  engaging  nearly 
all  staff  in  the  operation  of  the  center,  it  has  brought  everyone  in  FVAP  closer  to  the 


5  In  Chapter  Two,  we  noted  that  the  communication  service  group  carried  responsibility  for  messaging  across 
the  agency  but  that  it  did  not  appear  to  be  as  actively  involved  in  developing  voting  and  election-official  assis¬ 
tance  guidance  and  training  materials,  all  of  which  contribute  to  the  public  face  of  FVAP.  We  argued  that,  if  the 
agency’s  activities  were  more  fully  integrated,  materials  speaking  to  mission  or  policy — regardless  of  their  origin 
or  purpose — could  be  honed  for  consistency  before  dissemination. 

6  The  rebranding  from  call  center  to  voting  assistance  center  reflected  an  intentional,  ongoing  migration  to 
fuller  service  and  occurred  in  the  summer  of  2013,  before  the  center  was  brought  in  house. 


Figure  7.5 

Federal  Voting  Assistance  Program  Organization  Chart  as  of  February  2015 


□  Permanent  position,  vacant  Voting  Assistance  Division 


SOURCE:  RAND  staff,  adapted  from  chart  provided  by  FVAP  leadership. 

NOTE:  FVAP  provided  updated  information  in  February  2015,  which  we  included  in  this  figure. 
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agency’s  mission.7  Moreover,  by  creating  a  readily  accessible  database  on  customer  con¬ 
tact  that  includes  information  on  the  substance  of  each  contact  and  customer  feedback, 
FVAP  has  begun  to  address  concerns  about  common  strategies  and  shared  methods  of 
engagement.  In  conjunction  with  staff  training  on  problem-solving  and  data  analysis, 
which  we  discuss  later  in  this  chapter,  the  agency  might  be  better  able  to  identify  cross¬ 
cutting  issues,  look  for  and  track  patterns  among  emerging  issues,  and  develop  unified, 
agency-wide  responses. 

Admittedly,  the  voting  assistance  center  does  not  handle  a  large  volume  of 
inquiries — it  received  almost  1,000  inquiries  from  June  2014  to  August  20148 — but 
the  reconfiguration  holds  potential  as  a  signal  that  FVAP  is  focusing  on  assistance, 
as  a  means  to  unite  the  agency  around  a  common  purpose,  and  as  an  opportunity 
to  gain  a  better  understanding  of  stakeholders’  needs.  In  addition,  the  center  might 
be  bringing  FVAP  closer  to  a  relatively  inaccessible  segment  of  the  UOCAVA  voting 
population — namely,  overseas-citizen  voters.  In  August  2014,  roughly  one-fifth  of  all 
inquiries  were  from  overseas-citizen  voters,  which  was  a  slightly  larger  share  than  the 
share  from  military  voters.  One  might  expect  that  overseas  citizens  would  account  for 
a  larger  share  of  the  center’s  clients  than  service  members  because  there  might  be  more 
of  them  and  they  might  lack  immediate  access  to  VAOs  and  other  installation-based 
voting  assistance;  however,  as  depicted  in  Figure  6.1,  they  also  benefit  from  the  assis¬ 
tance  of  overseas  NGOs. 

Further  reflecting  FVAP’s  intent  to  assist,  the  agency  is  exploring  new,  more- 
targeted  modes  of  engaging  directly  with  states  and  localities,  including  through  in- 
depth  conversations  and  troubleshooting  sessions  and  work  with  CSG.9  Regarding 
direct  engagement,  FVAP  recently  met  with  election  officials  in  Austin,  Texas,  at  a 
round-table  event  held  on  the  margins  of  the  Texas  Election  Law  Seminar.  It  used  the 
opportunity  to  brainstorm  with  the  officials  on  Texas-specific  issues  and  to  engage 
on  other  matters  of  interest  around  the  table,  including  ballot  rejection  rates.  CSG 
and  FVAP  have  embarked  on  a  cooperative  agreement,  through  which  FVAP  hopes 
to  leverage  CSG’s  links  to  the  states  to  address  issues  at  the  intersection  of  their  con¬ 
cerns  about  meeting  UOCAVA  requirements.10  To  that  end,  CSG  has,  with  input  from 


7  VAOs  and  voters,  the  latter  consisting  of  a  combination  of  overseas-citizen  voters,  military  voters,  and  a  small 
number  of  non-UOCAVA  voters,  accounted  for  roughly  similar  shares  of  the  center’s  clients  in  August  2014,  with 
election  officials  accounting  for  the  next-largest  share.  On  that  basis,  the  center  might  be  said  to  serve  interme¬ 
diaries  somewhat  more  often  than  voters.  Such  an  observation  would  be  consistent  with  the  analysis  in  Chapter 
Six  of  FVAP’s  role  in  the  voting  system,  which  characterized  FVAP  as  often  depending  on  or  operating  through 
intermediaries. 

8  FVAP  reported  997  calls  from  June  2,  2014  to  August  26,  2014,  of  which  more  than  one-third  occurred  in 
August,  but  FVAP  anticipated  a  higher  rate  of  inquiries  in  October,  in  the  run-up  to  the  November  elections. 

9  In  the  next  section,  we  discuss  some  of  the  implications  of  the  approach  for  stakeholder  engagement. 

10  As  a  related  matter,  it  is  our  understanding  that  FVAP  is  working  with  a  contractor  to  develop  a  checklist  of 
requirements  for  election  officials,  suggesting  another  step  toward  meeting  the  mandate  to  ensure  their  awareness 
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FVAP,  established  two  working  groups,  one  looking  at  technology-related  issues  and 
the  other  at  policy-related  issues.  Initial  topics  of  discussion  in  those  working  groups 
have  included  data  standardization  and  the  use  of  forms  as  UOCAYA  voter-registra¬ 
tion  applications. 

FVAP  is  also  reassessing  DoDI  1000.04  with  the  intent  to  craft  revised  guidance 
that  is  in  closer  alignment  with  core  legal  requirements  and  more  clearly  sets  out  the 
responsibilities  of  each  participant  in  the  system  as  part  of  a  system.  The  agency  is  map¬ 
ping  each  provision  of  DoDI  1000.04  to  UOCAVA  and  vice  versa  to  promote  both 
consistency  and  completeness  in  the  new  version.  It  is  also  drawing  together  each  insti¬ 
tution  and  individual’s  (e.g.,  IVAO’s,  UVAO’s)  responsibilities  under  a  single,  readily 
locatable  heading.* 11 

Lastly,  we  note  a  substantial  change  in  FVAP’s  management  of  professional  devel¬ 
opment  and  of  management  itself.  FVAP  is  adopting  a  routinized  approach  to  working 
with  staff  to  identify  and  fill  training  needs,  to  replace  what  had  been  described  as  an 
ad  hoc  arrangement.  FVAP  leadership  has  assigned  this  responsibility  to  a  member  of 
the  mission-support  team  and,  in  a  manner  that  is  consistent  with  our  recommenda¬ 
tions  on  capability  and  capacity,  it  is  also  providing  that  team  member  with  training, 
e.g.,  an  initial  course  titled  “Supporting  Professional  Growth  in  Organizations,”  to  do 
the  job.  At  the  same  time,  FVAP  is  investing  in  its  managers  and  management  with 
courses  in  organizational  and  project  management,  ranging  from  “Leadership  and 
Management  Skills  for  Non-Managers”  to  “Administration  in  Public  and  Non-Profit 
Organizations.”  FVAP  also  had  one  GS-14-level  staff  member  scheduled  to  attend  a 
defense  senior  leadership  development  program  (the  Key  Executive  Leadership  Certifi¬ 
cate  Program),  starting  early  in  the  2015  fiscal  year.12 

Table  7.2  presents  FVAP’s  training  commitments  from  spring  of  2014  to  the  pres¬ 
ent,  exclusive  of  the  senior  leadership  commitment,  noted  above. 


Build  Trust  and  Strengthen  Relationships 

Building  trust  and  strengthening  relationships  will  take  time,  but  FVAP  appears  to  be 
progressing  in  its  approach  to  working  with  partners  and  serving  customers,  commu¬ 
nicating  with  stakeholders,  and  promoting  transparency. 


of  their  requirements  under  UOCAVA.  As  addressed  in  Chapter  Four,  the  law  (42  U.S.C.  §  1973ff[b][l])  includes 
a  general  requirement  for  FVAP  to  consult  state  and  local  election  officials  in  carrying  out  UOCAVA  and  ensure 
their  awareness  of  UOCAVA  requirements. 

11  The  September  2012  version  of  DoDI  1000.04  delineates  responsibilities  by  institution,  but  the  responsibili¬ 
ties  of  people  within  those  institutions  are  scattered  across  subsections. 

12  One  staff  member  participated  in  a  defense  senior  leadership  program  in  the  fall  of  2013. 
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Table  7.2 

Federal  Voting  Assistance  Program  Professional  Development  Commitments 


Date  (2014) 

Employee  Grade 

Course  Name 

Course  Hours 

March  1 

GS-12 

Election  and  Voter  Registration  Administration 

16 

March  1 

GS-12 

Election  Center  Voter  Registration 

16 

March  17 

GS-15 

Strategic  Workforce  Planning 

16 

April  26 

GS-12 

Information  Management  and  Technology 

12 

May  17 

GS-15 

Administration  in  Public  and  Nonprofit  Organizations 

40 

May  20 

GS-12 

Developing  Website/[Cascading  Style  Sheet] 

32 

May  30 

GS-14 

Financial  Management  Professional  Development 

32 

June  9 

GS-12 

Project  Management  Principles 

21 

July  21 

GS-12 

Leadership  and  Communication  Skills  for  Project 
Managers 

21 

July  31 

GS-14 

Monitoring  Grants 

16 

August  13 

GS-12 

Microsoft  Project  2010  Introduction  Course 

16 

August  19 

GS-12 

Comparing  Election  Performance  by  Measuring 
Processes  and  Results 

24 

August  20 

GS-12 

Leadership  and  Management  Skills  for  Non-Managers 

24 

August  25 

GS-15 

Problem  Solving  and  Data  Analysis 

40 

September  2 

GS-13 

Introduction  to  Geographic  Information  Systems 

16 

September  5 

GS-14 

Supporting  Professional  Growth  in  Organizations 

8 

September  11 

GS-12 

Customer  Service  Skills  and  Techniques3 

16 

September  16 

GS-12 

Management  Scope,  Schedule,  and  Cost 

21 

September  18 

GS-12 

Appropriations  Law  Refresher 

16 

September  19 

GS-14 

4  Steps  to  Standout  Legal  Writing 

6 

September  22 

GS-14 

Strategic  Plans  to  Tactical  Execution 

18 

September  23 

GS-12 

Six  Sigma  White  Belt  Training 

5 

September  24 

GS-13 

Leadership  and  Management  Skills  for  Non-Managers 

24 

September  30 

GS-14 

Effective  Writing  for  Lawyers  Workshop 

6.5 

October  6 

GS-14 

Six  Sigma  Green  Belt  Training 

80 

October  7 

GS-12 

Six  Sigma  Yellow  Belt  Training 

5 

SOURCE:  Based  on  FVAP  staff  communication,  September  9,  2014. 

NOTE:  All  of  the  sessions  included  in  this  table  were  fully  funded  and  approved. 
a  Train-the-trainer  event,  with  agency-wide  training  scheduled  for  September  15,  2014. 
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In  this  section,  we  draw  further  evidence  from  the  redesign  of  the  FVAP  website, 
the  reconfiguration  of  the  call  center,  new  forms  of  engagement  with  states,  and  com¬ 
mitments  to  professional  development,  and  we  point  to  a  recent  NASS  resolution  on 
voting  information,  the  continuation  of  the  OMB  process  for  form  revisions,  feedback 
from  the  voting  assistance  center,  and  the  development  of  a  new  product  line,  i.e.,  the 
research  note. 

First,  turning  to  the  FVAP  website,  we  see  signs — in  the  before  and  after  screen- 
shots  presented  above  and  in  our  own  exploration  of  the  tabs — of  substantial  improve¬ 
ment  in  the  usability  of  the  agency’s  website  for  non-UOCAVA  voter  stakeholders.  In 
prior  conversations,  stakeholders  had  credited  FVAP  with  developing  a  well-appointed 
product  for  voters  but  suggested  the  need  for  enhancements  for  other  stakeholders. 
The  new  home  page  displays  tabs  for  military  voters,  overseas-citizen  voters,  VAOs, 
and  election  officials  in  a  single  banner  that  also  contains  a  general  information  tab 
with  access  to  information  about  FVAP,  reports  and  surveys,  and  outreach  materials.13 
The  home  page  also  includes  upcoming  election  dates,  quick  links,  latest  FVAP  news, 
a  “contact  FVAP”  feature,  and  rotating  images,  with  clickable  links  to  a  statement 
from  the  Secretary  of  Defense,  information  on  the  use  of  online  voting  tools,  informa¬ 
tion  specific  to  overseas- citizen  voters  and  VAOs,  and  information  for  voters  who  have 
moved  recently.14 

As  noted  above,  FVAP  has  a  new  page  devoted  to  outreach  materials  that  were 
previously  scattered  across  the  website.  We  view  the  page  as  having  implications  for 
how  FVAP  can  and  might  do  business  with  its  stakeholders.  For  example,  the  materi¬ 
als  include  a  “blogger  widget,”  ready  to  cut  and  paste.  The  intent  is  to  make  it  easier 
for  partners  to  link  voters  to  FVAP  resources  and,  thus,  to  leverage  their  reach.  Simi¬ 
larly,  FVAP  has  attempted  to  extend  its  reach  through  distribution  of  outreach  kits  to 
DoDEA  schools.15 

But  not  all  of  the  new  or  revised  features  are  reaching  their  potential,  with  some 
shortcomings  suggesting  need  for  either  relabeling  or  more-active  upkeep,  for  technical 
corrections,  and  for  additional  links.16 

•  On  September  15,  2014,  both  the  “upcoming  election  events”  and  the  “latest 
FVAP  news”  sections  of  the  home  page  appeared  to  be  lacking  up-to-date  infor¬ 
mation  relevant  to  the  November  election.  Upon  discussion  with  FVAP,  we 
learned  that  the  calendar  and  the  news  listings  update  automatically  on  a  set 


13  This  banner  appears  on  all  the  FVAP  web  pages. 

14  Current  on  the  home  page  as  of  September  15,  2014. 

15  FVAP  is  also  discussing  the  provision  of  a  curriculum  for  high  school  seniors. 

16  We  did  not  attempt  a  comprehensive  assessment  of  the  website;  rather,  we  comment  on  issues  that  caught  our 
attention  for  better  or  worse  in  our  search  for  evidence  of  change. 
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schedule  and  that  the  latter  draws  items  from  FVAP’s  email  alerts  but  not  neces¬ 
sarily  from  broader  news  sources. 

•  The  reports  and  surveys  are  now  intended  to  be  searchable,  both  in  bulk  and 
under  the  subcategories  “Reports  to  Congress,”  “Surveys,”  and  “General  Research 
Reports,”17  but  our  attempts  at  subcategory  searches  did  not  proceed  smoothly.18 
F  VAP  staff  was  aware  of  the  problem  at  the  time  of  our  analysis  and  was  seeking 
a  remedy. 

•  FVAP  provides  links  to  reports  from  other  U.S.  government  agencies,  consisting 
of  GAO,  EAC,  and  the  DoD  IG,  but  it  does  not  include  links  to  other  federal 
partners’  reports,  such  as  those  of  NIST,19  DOJ,  or  MPSA. 

FVAP  has  also  introduced  a  new  product,  the  research  note,  to  its  product  slate. 
According  to  FVAP,  this  product  line  is  intended  to  bring  technical  information  and 
research  Endings  to  the  general  public  in  a  more  focused  and  readable  form  than  in 
larger,  full-blown  research  reports.  And,  it  appears  to  do  so.  The  first  such  note  (FVAP, 
2014b),  “Registration  and  Voting  Participation  Differences  between  the  Active  Duty 
Military  and  Citizen  Voting  Age  Population,”  also  represents  progress  toward  method¬ 
ological  transparency.20 

In  the  prior  section,  we  discussed  the  reconfiguration  of  the  call  center  as  a  matter 
of  mission;  here,  we  consider  it — and  the  feedback  that  it  has  generated — as  evidence  of 
an  ongoing  effort  to  build  trust  and  strengthen  relationships  with  stakeholders  and  as  a 
concrete  manifestation  of  service  to  customers.  FVAP’s  emphasis,  thus  far,  has  been  on 
firming  up  its  standing  both  as  a  trusted  source  of  information  about  UOCAVA  voting 
and  as  a  responsive,  customer  service-oriented  entity.  In  support  of  this  effort,  FVAP 
is  also  investing  in  professional  development,  consisting  of  an  introductory  brown-bag 
session  on  the  operation  of  the  center,  staff  enrollment  in  a  train-the-trainer  session  on 
customer  service  skills  and  techniques,  and  a  subsequent  agency-wide  training  session. 
FVAP  has  also  produced  a  “standards  and  procedures”  volume  for  use  by  those  staffing 
the  center. 

In  June  2014,  FVAP  began  surveying  its  call-center  customers  and,  on  the  basis 
of  the  results  of  those  surveys  and  a  small  amount  of  free-form  feedback,  this  practice 
appears  to  be  a  positive  development.  Over  a  roughly  three-month  period,  beginning 
in  June  2014,  FVAP  assisted  almost  1,000  customers,  consisting  of  VAOs,  overseas-cit- 


17  Prior  to  the  redesign,  a  “reference  and  reports”  tab  was  located  on  the  home  page,  not  via  “General  Informa¬ 
tion,”  but  the  material  was  not  searchable,  and  finding  specific  items  was  more  cumbersome. 

18  Clicking  on  “General  Reports”  took  us  to  a  mixed  list  of  all  categories  of  reports,  which  reduced  the  visibility 
of  the  general  reports  and  could  make  it  difficult  to  find  the  general  material. 

19  As  of  September  17,  2014,  NIST  offered  a  wide  range  of  materials  potentially  relevant  to  those  with  an  interest 
in  UOCAVA  voting  (NIST,  2015). 

20  FVAP  has  since  released  a  second  note  in  this  series;  see  FVAP,  2014a. 
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izen  voters,  military  voters,  election  officials,  and  others.  Of  the  total,  FVAP  surveyed 
214  customers  and  obtained  29  responses.21  Customers  were  asked  to  rate  FVAP  along 
four  lines — specifically,  efficiency  in  resolving  the  request,  courtesy  and  professional¬ 
ism,  skill  level  of  the  analyst,  and  overall  satisfaction  with  FVAP.  On  a  scale  of  1  to  5, 
its  average  ratings  along  each  line  ranged  from  4.7  to  4.8.  Admittedly,  there  were  few 
respondents,  but  the  results  are  encouraging.  FVAP  also  received  open-ended  feedback 
from  ten  customers  during  the  same  period.  That  feedback  was  almost  entirely  positive, 
e.g.,  citing  quick  turnarounds,  relevant  responses,  and  personalized  service. 

Here,  we  also  draw  from  the  prior  discussion  of  FVAP’s  engagement  with  states 
and  localities,  both  through  targeted,  direct  contact,  as  in  the  case  of  the  Texas  round 
table,  and  through  the  CSC  cooperative  agreement.  In  the  Texas  example,  we  detected 
a  subtle  but  important  change  of  tone  and  a  less  subtle,  more  readily  verifiable  change 
in  methods.  Our  discussions  with  FVAP — and  the  content  of  an  FVAP-drafted  meet¬ 
ing  synopsis — strongly  suggest  that  FVAP  entered  the  event  seeking  not  just  to  broad¬ 
cast  but  also  to  receive  and  respond.  FVAP  annotated  the  synopsis  with  action  items 
for  FVAP,  and,  in  a  break  from  past  behavior,  the  agency  set  about  taking  action  and 
following  up  with  participants  to  alert  them  to  that  action.  In  particular,  the  FVAP 
director  used  a  DoD  news  event  to  offer  absentee-voting  tips  to  service  members, 
including  two  that  emerged  from  the  round  table,  i.e.,  using  state  forms  when  voting 
locally  and  remembering  to  check  inboxes  for  ballots  when  requesting  to  receive  them 
by  email.22  Through  CSG,  FVAP  also  appears  to  be  addressing  its  relationships  dif¬ 
ferently.  Notes  from  recent  meetings  suggest  that  FVAP  is  giving  the  states — or  CSG, 
serving  as  the  coordinator  of  two  working  groups — considerable  latitude  in  selecting 
topics  and  guiding  discussions.  This  is  not  to  say  that  FVAP  is  not  participating  but 
that  it  does  not  appear  to  be  attempting  to  control  the  dynamic. 

Suggesting  an  increase  in  trust  in  FVAP  as  a  partner — and  increased  confidence 
in  the  FVAP  website — NASS  issued  an  amended  “NASS  Resolution  on  Voting  Infor¬ 
mation  for  Military  and  Overseas  Citizens  on  the  Web,”  in  July  2014.  That  resolution 
included  the  following  language: 

Now  therefore  be  it  resolved,  that  NASS  requests  that  each  state’s  elections  Web 
site  provide  specific  information  for  UOCAVA  voters,  and  that  each  Web  site  pro¬ 
vide  a  link  to  the  U.S.  Department  of  Defense  Federal  Voting  Assistance  Program 
Web  site. 


21  The  overall  response  rate  for  the  three-month  period  was  about  14  percent  but  appears  to  have  been  increasing 
during  that  period.  FVAP  noted  that  the  initial  response  rate  was  only  7  percent  and  that  it  had  taken  remedial 
action,  drawing  on  reported  best  practices,  to  improve  the  rate,  e.g.,  by  delaying  the  survey  request. 

22  Quoting  the  director,  “Don’t  use  the  federal  postcard  application  if  you  plan  on  voting  in-person  at  the  polls. 
Instead,  use  your  state  form.  Also,  don’t  forget  to  check  your  inbox  for  your  ballot  if  you’ve  requested  to  receive  it 
by  e-mail”  (Marshall,  2014). 
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We  conclude  this  section  with  a  brief  discussion  of  FVAP’s  use  of  the  OMB  pro¬ 
cess  for  revising  the  FPCA  and  FWAB.  In  Chapter  Three,  “Stakeholder  Outreach,”  we 
summarized  a  comparison  of  two  recent  efforts  (2011  and  2013)  to  revise  the  FPCA 
and  FWAB  forms.  Discussants  depicted  the  first  effort,  led  by  DoD,  as  one  in  which 
F  VAP  staff  did  not  solicit  or  respond  uniformly  to  external  input,  resulting  in  animos¬ 
ity  among  some  stakeholders.  They  depicted  the  second  effort,  run  through  OMB,  as 
open,  constructive,  and  yielding  buy-in.  They  did  not  suggest  that  everyone’s  needs 
had  been  met  through  the  OMB  process — or  that  they  would  stop  advocating  for  their 
concerns — but  that  they  had  been  heard  and  understood  the  reasons  for  the  outcome 
of  the  process.  Discussants  held  up  the  OMB  process  as  a  positive  example  of  transpar¬ 
ency,  listening,  and  responding. 

Since  conducting  that  outreach,  we  have  gathered  more  evidence  on  the  OMB 
process  and  FVAP’s  use  of  it  and  have  verified  that  the  process  did  yield  systematic 
engagement,  far  beyond  what  occurred  in  the  prior  revision  of  the  FPCA  and  FWAB.23 
Over  two  cycles,  involving  opportunities  to  comment  and  reply,  FVAP  fielded  ques¬ 
tions,  comments,  and  concerns  on  usability,  layout,  additional  blocks  and  space  for 
signatures,  instructions,  and  specific  block  changes,  from  stakeholders  associated  with 
at  least  61  comment  identification  numbers;24  posted  a  revised  draft;  and  finalized  the 
forms.25  To  supplement  the  process,  FVAP  reached  out  separately  to  stakeholders  to 
alert  them  to  the  process  and  posted  news  releases,  similarly  announcing  the  process 
(see  FVAP,  2013a,  2013c).  The  process  took  five  months  from  start  (April  1,  2013)  to 
finish  and,  by  all  accounts,  yielded  an  improved  product.26 

We  offer  this  as  an  example  of  FVAP  taking  steps,  on  its  own,  to  identify  better 
ways  of  working  with  stakeholders — and  doing  business  more  generally.  FVAP  chose 
to  make  use  of  the  OMB  process  prior  to  our  arrival  and,  recognizing  the  value  of  that 
process,  plans  to  make  use  of  it  in  the  future. 


Embrace  a  Culture  and  Principles  of  Effectiveness 

The  evidence  suggests  progress  in  several  areas.  First,  as  noted  above,  FVAP  has  imple¬ 
mented  change  on  the  basis  of  evidence;  that  is,  it  established  its  new  statement  of  pur¬ 
pose,  undertook  a  substantial  reorganization,  and  reoriented  its  operations  on  the  basis 


23  In  2011,  FVAP  posted  the  forms  in  the  Federal  Register  and  commissioned  a  usability  study  but  did  not  issue 
an  open  call  for  comment  or  actively  seek  widespread,  if  any,  stakeholder  input. 

24  We  tallied  40  comment  identifiers  associated  with  comments  on  the  FPCA  and  21  comment  identifiers  associ¬ 
ated  with  comments  on  the  FWAB,  but  there  might  have  been  overlap  across  the  two  sets. 

25  These  comments  were  associated  with  the  first  60-day  cycle. 

26  Notices  for  the  proposed  forms  (FWAB  and  FPCA)  and  requests  for  comment  were  posted  on  April  1,  2013 
(Office  of  USD[P&R],  2013a,  2013b). 
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of  stakeholder  outreach,  statistical  evidence  of  efficacy,  the  analysis  that  we  provided  to 
the  agency  in  phase  1  of  this  project,  and  the  final,  robust  logic  model.  For  example, 
FVAP  drew  directly  from  the  statistical  analysis  presented  in  the  2012  postelection 
report  to  Congress  (FVAP,  2013b)  in  its  decision  to  focus  attention  on  communica¬ 
tions  and  outreach  as  means  to  build  voters’  awareness  of  FVAP  resources.  That  analy¬ 
sis  suggested  the  importance  of  interaction  with  FVAP  resources,  which,  in  turn,  sug¬ 
gested  the  necessity  of  voters’  awareness  of  those  resources.  A  heavy  reliance  on  analysis 
was,  in  itself,  a  major  change  in  FVAP’s  approach  to  setting  course  and  taking  action. 

Second,  FVAP  commissioned  the  development  of  the  dashboard — in  a  related 
RAND  project — to  provide  indicators  of  needs  throughout  the  voting  assistance 
system  and  track  progress  toward  meeting  those  needs.  Moreover,  with  input  from 
that  project,  FVAP  has  recently  redesigned  its  VAO  and  IVA  office  metric-reporting 
requirements.  The  new  requirements  include  an  updated  list  of  metrics  and  the  justi¬ 
fication  for  each  metric.  Examples  of  justifications  include  accurately  representing  the 
use  of  particular  resources  and  characterizing  preferred  methods  of  distribution  of  par¬ 
ticular  resources.  FVAP  has  also  expressed  its  intent  to  use  the  dashboard  to  identify 
hot  spots  and  better  target  its  2016  VAO  training  program. 

Third,  FVAP  is  encouraging  and  supporting  enrollment  in  relevant  professional 
development,  including  courses  in  organizational  and  project  management,  strategic 
planning  and  tactical  execution,  performance  measurement,  and  problem-solving  and 
data  analysis. 

Fourth,  a  small  number  of  FVAP  employees  has  expressed  interest  in  using  the 
project-management  tools  that  the  RAND  project  team  provided  in  phase  2  of  this 
project  to  explore  the  need  for  initiatives  relating  to  state  engagement  and  HR  manage¬ 
ment  and  to  evaluate  their  potential  benefits,  costs,  and  risks. 

Fifth,  implicit  in  the  shift  to  certain  priorities  is  a  shift  away  from  others,  both  in 
substance  and  in  the  allocation  of  resources.  FVAP  appears  to  be  letting  go  of  activi¬ 
ties  that  are  not  consistent  with  its  reinvention  and,  given  plausible  capabilities  and 
capacities,  unlikely  to  serve  voters’  interests  effectively.  For  example,  in  shifting  toward 
troubleshooting  with  the  states,  as  it  did  in  Texas,  it  has  shifted  away  from  legislative 
initiatives.  Lastly,  we  point  to  the  right-sizing  that  has  accompanied  FVAP’s  reorgani¬ 
zation;  for  example,  since  the  inception  of  this  project,  FVAP  has  relinquished  all  of 
its  term  positions,  some  through  conversion  to  permanent  FTE  positions  and  others 
through  elimination. 


Concluding  Remarks  on  Evidence  of  Change 

The  foregoing  evidence  suggests  both  real  change  and  progress  toward  change.  FVAP 
has  instituted  organizational  and  operational  change  to  bring  the  agency  in  closer 
alignment  with  core  requirements  and  with  a  mission  oriented  toward  more-direct, 
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hands-on  assistance  to  intermediaries  and  voters.  We  cannot,  as  yet,  connect  those 
changes  to  changes  in  outcomes  for  UOCAVA  voters — the  changes  in  FVAP  are  too 
recent — but  we  see  potential  to  do  so,  eventually  with  the  assistance  of  the  metrics  and 
measures  from  the  dashboard. 

The  evidence  of  shortcoming — such  as  those  found  on  the  website — also  suggests 
the  need  to  keep  moving  forward,  primarily  to  lock  in  and  build  on  gains,  but  also  to 
correct  occasional  missteps.  In  the  next  chapter,  we  offer  suggestions  for  doing  so. 


CHAPTER  EIGHT 


Final  Recommendations,  Guidance,  and  Remarks 


In  this  chapter,  we  offer  our  final  recommendations  and  guidance  to  FVAP.  We 
reported  evidence  of  positive  change  in  Chapter  Seven,  but  we  also  see  the  potential 
for  both  additional  gains  and  fatigue  in  the  future.  For  those  reasons,  our  final  recom¬ 
mendations  and  guidance  speak  to  locking  in  and  building  on  recent  progress.  We  also 
consider  FVAP’s  absorptive  capacity — that  is,  its  ability  to  preserve  momentum  with¬ 
out  burning  out.  We  conclude  the  chapter  with  remarks  that  include  a  look  back  to  the 
project’s  goals  and  comments  on  progress  in  meeting  them. 


Lock  in  and  Build  on  Gains 

FVAP  needs  to  adhere  to  its  mission,  notwithstanding  the  near  certainty  of  some  lead¬ 
ership  turnover,  the  push  and  pull  of  divergent  stakeholder  interests,  and  the  ebbs 
and  flows  of  election  cycles,  but  we  are  not  advocating  rigidity.  FVAP  must  be  able 
to  respond  to  emerging  needs,  some  foreseeable  and  some  not;  it  is  through  those 
responses  that  it  will  demonstrate  whether  it  knows  what  it  is  and  where  it  is  going. 

What  follows  are  suggestions  for  consolidating  and  advancing  recent  progress. 
We  have  drawn  these  suggestions  from  our  observations  of  FVAP  and  its  stakeholders 
and,  informally,  from  the  literature  on  change  management  (see  Camm  et  ah,  2013, 
for  a  synthesis)  and  organizational  resilience  (e.g.,  Bhamra,  Dani,  and  Burnard,  2011; 
Vogus  and  Sutcliffe,  2007). 1  Some  of  these  suggestions  are  within  FVAP’s  power  to 


1  Here,  we  draw  loosely  from  two  research  streams:  the  first  on  change  management  (for  a  succinct  overview, 
see  Camm  et  ah,  2013,  pp.  87-95)  and  the  second  on  organizational  resilience.  In  referring  to  the  literature  on 
change  management,  we  draw  heavily  from  Camm  et  al.’s  synthesis  but  also  from  Kotter,  2007.  To  the  extent  that 
we  address  resilience,  we  recognize  the  existence  of  a  vast  literature,  offering  myriad  definitions.  For  purposes  of 
this  report,  we  adopt  the  conceptualization  of  Vogus  and  Sutcliffe,  2007,  p.  3418,  which  defines  resilience  as  “the 
maintenance  of  positive  adjustment  under  challenging  conditions  such  that  the  organization  emerges  from  those 
conditions  strengthened  and  more  resourceful.”  Among  the  “challenging  conditions,”  they  include  “discrete 
errors,  scandals,  crises,  and  shocks,  as  well  as  ongoing  risks  (e.g.,  competition),  stresses,  and  strain.”  The  potential 
for  strengthening  and  the  inclusion  of  ongoing  stresses  and  strains  sets  this  definition  apart  from  some  others. 
On  September  23,  2014,  a  search  for  organizational  resilience  in  Google  Scholar  yielded  about  141,000  results  in 
0.05  seconds — a  search  on  Google,  in  its  entirety,  yielded  about  3,340,000  results  in  0.17  seconds. 
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implement,  and  others  must  be  taken  up  elsewhere  in  DoD,  at  higher  organizational 
levels,  or  outside  DoD.  Turning  first  to  the  suggestions  largely  within  FVAP’s  control, 
we  urge  FVAP  to  continue  to  do  the  following: 

•  Internalize — and  outwardly  project — the  mission. 

•  Invest  in  leadership  and  staff  with  professional  development. 

•  Promote  organizational  cohesion. 

•  Engage  with  stakeholders  as  stakeholders. 

•  Take  calls  for  effectiveness  to  heart. 

•  Develop  stronger  discernment  skills. 

•  Work  with  the  logic  model  as  a  living  document. 

•  Undertake  periodic  health  checks. 

In  the  rest  of  this  section,  we  elaborate  on  each  point.  In  bulleted  text,  we  use 
recommendations  on  YAO  training — and  other  matters — to  highlight  specific 
opportunities. 

Internalize — and  Outwardly  Project — the  Mission 

FVAP,  consisting  of  its  leadership  and  staff,  must  operate  in  accordance  with  the  agen¬ 
cy’s  mission  and  present  it  consistently  as  part  of  the  agency’s  public  face  through  the 
website,  written  materials,  presentations,  and  actions.  Throughout  this  report,  we  have 
presented  the  agency’s  mission  as  a  center  from  which  change  could  flow  (Chapters 
Two  and  Five)  and  did  flow  (Chapter  Seven),  but  it  should  also  serve  as  an  anchor  in 
the  future.  An  FVAP  that  has  a  strong  sense  of  self  is  an  FVAP  that  is  less  likely  to 
be  buffeted  by  turnover,  competing  demands,  and  the  more-predictable  stresses  and 
strains  of  an  election  cycle. 

•  FVAP  has  reoriented  its  mission  and  is  working  toward  fully  absorbing  it,  but  the 
agency  has  not  yet  built  that  mission — and  the  corresponding  message — into  the 
VAO  training  program  (Chapter  Six),  nor  has  it  consistently  embedded  its  prin¬ 
ciples  in  the  pages  of  its  website  (Chapter  Seven).  In  particular,  FVAP’s  in-person 
training  modules — in  use  in  the  now-complete  2014  training  cycle — were  born 
of  the  old  FVAP,  not  the  new,  and  will  need  to  be  updated,  as  will  the  VAO- 
targeted  web  pages. 

•  To  further  promote  the  agency’s  adherence  to  its  mission  and  in  recognition  of 
the  essential  bond  between  the  agency’s  mission  and  the  core  requirements  set  out 
in  Chapter  Four,  we  suggest  again  (Chapter  Five)  that  FVAP  establish  clear  lines 
of  accountability  to  legislative  mandates.  The  agency  should  be  able  to  trace  each 
provision  of  law  to  a  person  with  primary  control  over  related  activities. 
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Invest  in  Leadership  and  Staff  with  Professional  Development 

FVAP  should  continue  to  provide  its  leadership  and  staff  with  the  professional  devel¬ 
opment  necessary  to  forward  the  mission.  Simply  put,  if  leadership  or  staff  members 
lack  the  knowledge  and  skills  to  do  their  jobs,  the  agency  as  a  whole  cannot  carry  out 
its  mission.  FVAP  has  made  substantial  advances  (Table  7.2  in  Chapter  Seven),  but  it 
will  need  to  continue  the  effort  and  should  do  so  systematically.  At  the  same  time,  it 
must  give  people  the  opportunity  to  absorb,  digest,  and  reflect  on  new  material  and  to 
identify  paths  to  incorporating  it  into  their  work  lives. 

•  FVAP  should,  as  planned,  undertake  routine  assessments  of  staff  members’  pro¬ 
fessional  development  needs,  but  it  should  also  conduct  need  assessments  from 
an  institutional  perspective — that  is,  with  consideration  to  FVAP-wide  gaps  in 
capabilities  and  capacities.  On  that  basis,  we  would  urge  FVAP  to  put  the  same 
amount  of  effort  into  developing  its  VAO  trainers — and  to  those  managing  the 
VAO  training  program — as  it  has  into  building  other  capabilities,  e.g.,  around 
management  and  analysis. 

Promote  Organizational  Cohesion 

FVAP  must  continue  to  give  meaning  to  the  integrative  process  that  began  with  the 
reorganization  of  divisions  and  activity  streams.  To  be  more  than  a  relabeling,  desk¬ 
shuffling  exercise,  leadership  must  manage  its  divisions  and  teams  as  parts  of  a  larger 
organization,  and  staff  must  perceive  that  they  are  part  of  that  organization.  For  lead¬ 
ership,  this  means  resisting  the  temptation  to  create  new  stovepipes,  accounting  for 
the  interrelatedness  of  actions  among  activity  streams,  and  recognizing  and  validating 
people  who  excel  in  this  environment — whether  through  staff  awards,  promotions,  or 
a  simple  “thank  you.”  And,  for  the  entire  agency,  it  means  better  communication  up 
and  down  and  within  and  across  activity  streams. 

•  In  an  organization  as  small  as  FVAP,  it  would  be  counterproductive  to  rely  on 
the  leaders  of  each  activity  stream  as  sole  nodes  of  communication  across  streams. 
FVAP  staff  should  be  encouraged,  through  a  combination  of  formal  and  informal 
mechanisms,  to  share  their  experiences,  including  routine  and  exceptional  chal¬ 
lenges,  success  stories,  and  nascent  ideas  to  better  meet  customer  needs  across 
streams.  The  mechanisms,  some  of  which  are  already  in  use,  could  range  from 
retreats  to  elevator  chats. 

•  We  observed  (Chapter  Seven)  that  FVAP’s  Communication  and  Outreach  Divi¬ 
sion  is  coordinating  interactions  with  Congress  and  with  overseas  NGOs,  but  not 
with  the  uniformed  services  or  state  and  local  election  officials.  For  that  reason, 
the  agency  will  need  to  be  especially  attentive  to  the  quality  of  collaboration 
between  the  Communication  and  Outreach  Division  and  the  Voting  Assistance 
Division. 
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As  a  related  matter,  the  agency  cannot  be  dependent  on  a  single  person  to  carry 
the  agency  or  an  activity  without  support  from  others.  The  responsibilities  of  individ¬ 
ual  staff  members  must  be  delineated  clearly,  but  staff  must  be  pulled  into  and  join  the 
effort  as  coparticipants  and,  in  an  organization  with  fewer  than  20  permanent  FTEs, 
should  be  able  to  reach  across  the  agency  to  both  backfill  and  surge.  Requirements  for 
backfill  and  surges  also  imply,  as  addressed  in  Chapter  Five,  a  need  for  cross-training. 

Engage  with  Stakeholders  as  Stakeholders 

FVAP  must  continue  to  remember  that  its  stakeholders  have  interests  of  their  own  that 
merit  close  consideration  and  are  not  a  passive  audience  or  extension  of  the  agency’s 
operations.  In  some  regards,  this  might  be  FVAP’s  most  challenging  duty.  Doing  so 
requires  that  FVAP  more  fully  embrace  two-way  communication — it  is  making  prog¬ 
ress  (Chapter  Seven) — and  reach  out  to  the  civilian  public.  But,  it  must  do  so  in  a 
national  security  institution,  DoD,  that  values  caution  and,  in  some  instances,  might 
discourage  openness. 

•  FVAP  will  have  ample  opportunity  in  the  coming  year  to  strengthen  its  engage¬ 
ment  skills,  through  its  operation  of  the  voting  assistance  center,  its  work  with 
CSC,  and  revisions  and  updates  to  the  FPCA,  FWAB,  and  DoDI  1000.04. 
Whereas  two-way  communication  is  all  but  hard  wired  into  the  OMB-led  pro¬ 
cess  for  the  FPCA  and  FWAB,  it  is  not  as  unmistakably  embedded  in  the  process 
governing  the  redo  of  the  DoD  instruction.  The  latter  requires  some  prespeci¬ 
fied  coordination  across  stakeholders  but  could  more  readily  take  on  a  unilateral 
flavor.  As  such,  the  DoDI  update  will  afford  FVAP  an  opportunity  to  demon¬ 
strate  that  it  is  not  intent  simply  on  broadcasting  to  its  stakeholders  but  also  on 
receiving  information  from  them  and  responding  to  them.  Looking  further  into 
the  future,  to  the  2016  VAO  training  cycle,  we  envision  the  training  program — 
specifically,  the  in-person  program — as  an  opportunity  to  engage  constructively 
with  stakeholders  and  reap  some  of  the  ancillary  benefits  addressed  in  Chapter 
Six. 

Take  Calls  for  Effectiveness  to  Heart 

To  fully  embrace  effectiveness,  FVAP  will  need  to  consider  the  benefits,  costs,  and 
risks  of  its  actions  in  relation  to  its  mission,  requirements,  and  operations  on  a  daily 
basis — not  just  from  its  perspective  but  from  the  perspectives  of  its  customers,  part¬ 
ners,  and  other  stakeholders.  Taking  this  step  implies  a  change  in  mind-set  and  behav¬ 
ior.  Because  it  requires  access  to  alternative,  external  perspectives,  its  success  depends 
at  least  partly  on  the  agency’s  progress  on  two-way  communication. 

•  Much  of  Chapter  Six  and  Appendixes  D  and  E  concern  themselves  with  the 
application  of  effectiveness  tools  (benefit,  cost,  and  risk)  to  the  VAO  training  pro- 
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gram;  thus,  we  suggest  applying  those  tools  to  the  redesign  of  that  program  for 
the  2016  training  cycle  and  to  efforts  to  streamline  in-person  delivery.2 

Develop  Stronger  Discernment  Skills 

FVAP  must  become  more  adept  at  distinguishing  between  good  and  bad  ideas  for  new 
projects  or  activities,  whether  internally  or  externally  generated.  To  the  extent  that 
FVAP  takes  on  projects  or  activities  that  look  attractive  but  do  not  make  sense  in  rela¬ 
tion  to  its  mission,  its  requirements,  and  its  organizational  capabilities  and  capacity,  it 
is  allowing  itself  to  become  less  focused  and  more  susceptible  to  setbacks  and  surprises. 
We  also  suggest  strict  adherence  to  the  “do  it  well'’  principle;  that  is,  if  FVAP  lacks  the 
ability  or  cannot  acquire  the  ability  to  do  something  well,  it  should  not  do  it  unless  it 
must  (Chapter  Four).  FVAP  will  also  need  to  find  ways  to  say  “no”  to  new  ideas  with¬ 
out  dampening  staff  and  stakeholder  interest  or  creativity.  Those  who  propose  ideas 
that  do  not  advance  to  trial  or  implementation  should  not  be  made  to  feel  judged  or 
reluctant  to  float  new  ideas — nor  should  their  peers.3 

Work  with  the  Logic  Model  as  a  Living  Document 

We  suggest  keeping  the  final,  robust  logic  model  at  hand  as  both  a  reminder  of  institu¬ 
tional  identity  and  a  tool  for  communicating  with  staff  and  stakeholders  with  a  shared 
vocabulary,  but  we  also  encourage  FVAP  to  revisit  the  specification  and  details  of  the 
model  over  time.4  The  model,  which,  in  various  forms,  provided  the  foundation  for  our 
analysis  of  FVAP’s  operations,  a  blueprint  for  the  agency’s  reorientation  and  reorgani¬ 
zation,  and,  in  the  context  of  the  dashboard  project,  a  means  to  identify  metrics  and 
measures,  can  do  more  in  the  future.  For  example,  FVAP  can  use  its  model  to  walk 
staff  and  stakeholders  through  the  agency’s  approach  to  voter  assistance,  including  the 
nature  of  its  relationships  with  stakeholders,  to  explore  alternative  approaches,  and  to 
obtain  feedback.  If  FVAP  uncovers  a  better  way  of  advancing  its  purpose,  it  can  update 
the  model  to  reflect  the  change.  In  that  way,  the  model,  albeit  a  final  version  for  the 
purposes  of  this  analysis  and  structurally  robust,  should  be  thought  of  as  a  living  docu¬ 
ment.  Many  federal  agencies  post  their  logic  models  on  their  websites,  accompanying 
a  statement  of  purpose  on  an  “about”  page,  and  use  them  to  build  their  strategic  plans 
and  budget  requests. 


2  Training  will  not  stop  between  then  and  now,  but,  apart  from  the  training  that  IVAOs  provide  to  UVAOs,  on 
installations,  it  will  occur  largely  online. 

3  In  a  workplace  that  values  staff  participation  and  creativity,  we  envision  a  proliferation  of  ideas  more  than  a 
diminution  and  would  expect  to  see  FVAP  facing  this  issue  more  regularly. 

4  As  addressed  in  Appendix  A,  the  logic  model  can  be  used  to  provide  internal  and  external  audiences,  including 
program  partners,  legislators,  and  other  stakeholders,  with  a  clear  representation  of  the  program’s  operations  and 
intent.  It  can  also  be  used  to  delineate  a  program’s  boundaries  and  responsibilities,  thereby  clarifying  the  meaning 
of  impact  and  results  as  they  relate  to  the  program. 
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Undertake  Periodic  Health  Checks 

We  also  suggest  that  F  VAP  institute  an  ongoing  series  of  preprogrammed  health  checks, 
preferably  with  external  input,  potentially  to  include  DHRA  or  other  USD(P&R) 
management.  Such  checks  can  be  used  to  promote  progress  and  circumvent  the  poten¬ 
tial  for  fatigue  and  backsliding.  FYAP  should  put  these  checks  on  the  calendar  now,  as 
a  visible  reminder  of  the  need  for  persistence  and  to  prepare  for  them  as  forcing  events. 
The  logic  model  and  any  metrics  and  measures  that  flow  from  it  in  the  dashboard 
project  can  provide  a  basis  for  the  check.5  A  combination  of  qualitative  and  quanti¬ 
tative  evidence,  including  metrics  and  measures  from  the  dashboard  project,  can  be 
used  to  track  progress  toward  solidifying  change  within  FVAP  and,  eventually,  toward 
improving  voting  outcomes. 

Crosscutting  Recommendations  and  Observations 

Throughout  this  discussion,  we  have  assumed  the  interest  and  participation  of  FVAP’s 
stakeholders,  both  within  DoD  and  beyond  its  confines.6  For  example,  SVAOs  will 
need  to  play  an  active  part  in  streamlining  training,  by  providing  well-considered  lists 
of  candidate  training  sites,  and  D1TRA  might  need  to  engage  in  health  checks.  And, 
as  stressed  in  Chapter  Six,  other  agencies — ones  not  under  FVAP’s  control — can  have 
major  effects  on  outcomes.  Although  FVAP  cannot  forcibly  elicit  constructive  engage¬ 
ment,  it  can  take  steps  to  promote  it  by  creating  an  environment  in  which  stakeholders 
deem  their  participation  worthwhile.  To  that  end,  we  turn  back  to  the  fundamental 
issues  of  relationship-building  and  communication.  FVAP  must  convince  its  stake¬ 
holders  that  it  is  taking  their  views  into  account  and  that  their  participation  is  impor¬ 
tant  not  only  for  FVAP  but  in  their  own  interest.  If,  for  example,  an  SVAO  under¬ 
stands  that  FVAP  cannot  better  target  in-person  training  and,  thus,  reduce  costs  to  the 
services  without  his  or  her  input  in  the  selection  of  installations,  the  SVAO  might  be 
more  likely  to  participate. 

H  igher-level  interventions  might  be  necessary  under  some  circumstances.  Not 
every  issue  will  need  the  director’s  or  DITRA’s  involvement — and  pulling  them  in  too 
often  could  be  counterproductive — but  some  issues  might  require  policy  support  and 
merit  elevation. 

Our  last  words  on  locking  in  and  building  on  gains  concern  the  appointment  of 
FVAP’s  leadership — specifically,  its  director — and  speak  to  the  issue  of  organizational 
resilience.  The  director  of  FVAP  should  come  to  the  table  with  strong  administrative 
skills,  as  a  trained  leader.  If  not  already  well-versed  in  voting  assistance,  he  or  she  can 
learn  from  staff  with  deep  subject-matter  expertise.  FVAP  is  an  agency  that  will  con- 


5  For  an  example  of  the  use  of  the  logic  model  for  these  types  of  purposes,  see  Greenfield,  Williams,  and 
Eiseman,  2006,  and  Williams  et  ah,  2009. 

As  we  learned  through  our  early  outreach  to  stakeholders,  this  is  not  always  the  case. 
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tinue  to  benefit  from  agnostic  civilian  leadership,7  e.g.,  in  the  form  of  a  professional 
administrator.  Voting  is  an  inherently  political  construct,  subject  to  sharply  divergent 
views  on  the  means  of  conduct,  be  they  printed  or  electronic,  and  a  plethora  of  other 
hot-button  issues.  To  minimize  the  potentially  destabilizing  effects  of  leadership  change 
(Chapter  One)  and  to  reduce  stakeholders’  concerns  that  FVAP’s  engagement  is  driven 
too  much  by  the  agendas  of  individuals  and  not  enough  by  the  mission  of  the  agency 
(Chapter  Three),  we  suggest  prioritizing  managerial  expertise  in  the  hiring  decision. 

Without  intending  to  end  pessimistically,  we  point  out  that  the  vast  majority  of 
all  organizational  change  fails.  Studies  of  change  in  the  private  sector  report  an  ill- 
defined  failure  rate  of  about  70  percent.  The  sources  of  data  for  that  estimate  and  the 
meaning  of  failure  are  poorly  documented  (Breese,  2014),  but  the  evidence  is  indica¬ 
tive  of  the  difficulty  of  change.  To  the  extent  that  we  are  aware  of  reliable  predictors 
of  successful  change,  the  literature  suggests  the  importance  of  drawing  staff  into  the 
process  of  change;  creating  an  environment  that  makes  it  possible  to  implement,  sus¬ 
tain,  and  reward  change;  and,  as  change  unfolds  and  is  institutionalized,  maintain¬ 
ing  good  communication  throughout  the  organization.  Staff  members  need  to  know 
why  they  are  doing  what  they  are  doing,  how  their  efforts  relate  to  those  of  others  in 
the  organization,  and  whether  their  efforts  are  paying  off.  Kotter,  2007,  p.  100,  which 
identifies  eight  reasons  for  failure,  cites  “undercommunicating  the  vision  by  a  factor 
of  ten”  as  one  reason  and  offers  a  generalizable  warning,  “without  reliable  and  credible 
communication  and  a  lot  of  it,  the  hearts  and  minds  of  the  troops  are  never  captured.” 


Concluding  Remarks  on  Project  Goals  and  Results 

This  project  was  established  to  assist  FVAP  in  aligning  its  strategy  and  operations  to 
better  serve  its  mission  and  to  strengthen  FVAP’s  capacity  to  set  its  own  course,  greet 
change,  and  communicate  its  role  in  the  voting  community. 

Looking  back  over  the  past  two  years,  we  believe  that  FVAP  has  made  consider¬ 
able  progress.  FVAP  has  reoriented  its  focus  and  operations;  has  redesigned  its  web¬ 
site  and  outreach  materials  to  enable  and  communicate  the  change;  has  substantially 
altered  its  approach  to  stakeholders;  and  is  investing  in  its  staff — through  professional 
development — to  build  the  capabilities  and  capacity  to  support  the  reorientation. 

This  project  might  have  helped  to  catalyze  change  in  FVAP,  but  most  of  that 
change  occurred  outside  the  engagement,  suggesting  that  FVAP  has  the  ability  and 
intent  to  continue  implementing  and  institutionalizing  change.  FVAP  appears  to  be 
consolidating  and  advancing  recent  progress  and  has  provided  us  with  updates  on 
some  of  its  latest  initiatives  (e.g.,  relating  to  the  renovation  of  its  training  program 


7  Civilian  leadership  is  mandatory  under  DoDI  1000.04,  2012.  According  to  that  instruction,  the  USD(P&R) 
shall  “Ensure  that  the  Director,  Department  of  Defense  Human  Resources  Activity  (DoDHRA)  designates  a 
civilian  Director  of  the  FVAP.  .  .  .”  See  enclosure  2,  J  1(f),  p.  7  of  the  DoD  instruction. 


114  The  Federal  Voting  Assistance  Program  and  the  Road  Ahead 


for  state  election  officials)  that  postdate  our  involvement.  As  noted  in  Chapter  Seven, 
we  cannot,  as  yet,  connect  FVAP’s  reorientation  to  improvements  in  outcomes  for 
UOCAVA  voters — the  changes  in  FVAP  are  too  recent  and  largely  untested — but  the 
steps  that  FVAP  has  taken  to  realign  its  mission  and  operations  should  cut  a  clearer  path 
from  the  program’s  activities  to  outcomes  and,  thus,  better  support  those  outcomes. 

We  also  see  the  results  of  this  engagement  as  demonstrating  to  FVAP  and  other 
agencies  that  they  can  take  concrete  action  to  overcome  obstacles  to  change  and  place 
themselves  among  the  minority  of  organizations  that  implements  change  successfully. 
Organizational  change  is  difficult,  as  both  the  large  literature  on  change  management 
and  experience  suggest,  but  still  attainable.  The  collaboration  began  with  an  intense 
period  of  discussion  with  FVAP  around  the  terms  of  engagement,  and  it  required  sub¬ 
stantial  involvement  on  the  part  of  FVAP  staff,  including  meetings  at  all  levels  of  the 
organization  and  in  groups  ranging  in  size  from  few  to  many.  Inclusivity  and  expan¬ 
siveness  were  necessary  to  understanding  and  meeting  the  organization’s  needs,  but 
staff  involvement  implied  time  and  focus.  The  daily  challenges  of  implementation,  to 
which  we  are  less  able  to  speak  directly,  were  at  least  as  great  as  those  of  identifying 
gaps  and  disconnects  and  the  means  to  fill  them.  The  agency’s  commitment  to  change 
and  the  mutual  trust  that  an  ongoing,  collaborative  relationship  can  engender  were 
also  essential  to  progress.  The  project  description  envisioned  the  centrality  of  trust  in 
noting  the  necessity  of  “special  familiarity  with  the  sponsor’s  mission  and  a  close  work¬ 
ing  relationship  with  that  sponsor”  (RAND  Corporation,  2013). 
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Logic-Modeling  Workshop  and  Materials 


In  this  appendix,  we  present  the  set  of  information,  consisting  of  the  logic-modeling 
workshop  notes,  agenda,  and  discussion  points,  that  we  (the  RAND  project  team)  pre¬ 
pared  for  FVAP  for  the  workshop  that  we  led  held  on  May  1-2,  2013. 1 


Logic-Modeling  Workshop  Notes 

We  provided  the  following  notes  to  FVAP  as  leave-behind  material  to  accompany  the 
presentation  on  logic  modeling,  Logic  Modeling  101,  that  was  the  first  full  session 
of  the  logic-modeling  workshop,  held  on  May  1-2,  2013.  They  have  been  modified 
slightly,  e.g.,  by  adding  headings  and  eliminating  bold  type,  to  better  match  the  style 
of  this  report. 

Introduction 

Logic  modeling  is  at  least  as  much  about  “process”  as  it  is  about  “product.”  Logic 
models  can  serve  many  purposes.  Among  them,  they  can  be  used  to  articulate  the 
critical  path  by  which  a  program2  seeks  to  achieve  outcomes  and  attain  its  mission; 
construct  a  well-aligned  strategy  in  support  of  its  mission;  communicate  internally  and 
externally;  and  conduct  program  evaluations.3  In  the  course  of  building  a  model,  a 
program  can  also  develop  a  new  way  of  thinking  about  itself — especially,  the  relation¬ 
ship  between  its  activities  and  its  purpose — and  create  a  shared  vocabulary  with  which 
to  chart  its  future.  In  effect,  the  modeling  process  can  evoke  a  fundamental  change 


1  We  have  added  a  small  number  of  additional  citations,  references,  and  resources. 

2  The  concepts  of  logic  modeling  apply  broadly  to  programs,  agencies,  and  other  institutional  configura¬ 
tions,  including  operational  lines  and  initiatives  and  program  areas;  here,  we  follow  the  literature  and  refer  to 
“programs.” 

3  The  discussion  in  these  notes  draws  largely  from  Greenfield,  Williams,  and  Eiseman,  2006,  and  Williams 
et  ah,  2009.  For  more  on  the  use  of  logic  models,  see  McLaughlin  and  Jordan,  1999,  and  Wholey,  Hatry,  and 
Newcomer,  2010. 
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in  workplace  culture.  Arguably,  whether  the  model  lives  on  is  less  important  than 
whether  the  change  in  thinking  lives  on. 

As  a  practical  matter,  the  notes  that  follow  focus  mostly  on  the  “product.” 

Logic  Modeling  101 

The  logic  model  presents  the  program’s  “narrative”  or  “story.”4  It  typically  offers  a  sim¬ 
plified  visual  representation  of  a  program’s  operations,  starting  with  inputs  and  pro¬ 
gressing  to  activities,  outputs,  customers,  and  outcomes.  A  logic  model  may  also  link  a 
program’s  operations  (i.e.,  what  the  program  does  alone  or  with  others  to  fulfill  its  mis¬ 
sion)  to  its  strategy,  which  we  define  as  the  goals,  objectives,  and  performance  measures 
that  derive  from  and  support  the  program’s  mission.  Operations  involve  resources, 
actors,  and  events;  strategy  speaks  of  intentions. 

The  underlying  premise  of  the  model  is  one  of  postulated  “if,  then”  relationships 
that  connect  actions  to  outcomes.  “If  the  program  does  this,  then  this  should  or  could 
happen.”  Ultimately,  the  activities  undertaken  as  part  of  the  “if,  then”  chain  of  events 
can  help  to  fill  a  societal  need  or  solve  a  societal  problem.  Consider  a  notional  chain 
pertaining  to  FVAP:5 

•  If  FVAP  staff  provides  training  to  a  VAO,  then  the  VAO  will  gain  knowledge  of 
voting  rules,  procedures,  or  resources  that  can  be  used  to  assist  a  UOCAVA  voter. 

•  If  the  VAO  applies  that  knowledge  in  his/her  interactions  with  a  UOCAVA  voter, 
the  transfer  of  knowledge  might,  ultimately,  facilitate  a  successful  vote. 

•  It  might  be  difficult  to  tease  out  FVAP’s  contribution  to  the  successful  vote,  e.g., 
in  a  statistical  analysis,  but  it  is  still  the  case  that  FVAP  contributed  and,  thus, 
had  an  “impact.”6 

The  logic  model  can  be  used  to  draw  distinctions  between  the  “partners”  who 
work  with  FVAP  to  create  products  and  services  and  the  “customers”  who  use  those 
products  and  services.  A  program,  such  as  FVAP,  might  undertake  many  of  its  activi¬ 
ties  cooperatively,  with  other  programs  or  agencies,  which  might  affect  FVAP’s  opera¬ 
tions  through  direct  and  indirect  channels.  Some  or  all  of  those  channels  might  be 
represented  in  the  logic  model,  depending  on  their  importance.  FVAP’s  “customers” 
might,  for  example,  include  UOCAVA  voters,  VAOs,  SVAOs,  and  election  officials.  In 
some  instances,  however,  the  same  entity,  such  as  an  SVAO  or  election  official,  might 
serve  as  a  partner  in  one  context  and  as  a  customer  in  another.  The  preceding  example, 


4  The  word  “story”  is  a  terms  of  art  in  this  context.  See  McLaughlin  and  Jordan,  1999. 

4  Taylor-Powell  and  Henert,  2008,  illustrates  the  premise  with  a  series  of  very  simple  “if,  then”  chains  of  events 
that  address  personal  needs  and  problems  (for  example:  find  food  eat  food  feel  better). 

6  We  follow  Williams  et  al„  2009,  p.  xi,  in  its  use  of  “impact.”  That  study  defines  impact  as  the  contribution  of 
activities  to  desired  societal  outcomes,  such  as  improved  health,  environment,  economic,  and  social  conditions. 
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involving  VAOs,  also  serves  to  highlight  the  distinction  between  “intermediate  cus¬ 
tomers,”  such  as  the  VAO  who  is  in  training,  and  “final  customers,”  such  the  newly 
enabled  UOCAVA  voter,  and  between  intermediate  and  end  outcomes. 

In  brief,  the  following  are  features  of  a  logic  model: 

•  Inputs  are  resources  that  go  into  and  guide  program  activities. 

•  Activities  are  the  actions  that  the  program  undertakes  to  produce  outputs. 

•  Outputs  are  the  tangible  products  that  activities  generate. 

•  Partners  are  those  who  work  with  programs  to  conduct  activities  or  enable  out¬ 
puts. 

•  Customers  (intermediate  and  final)  are  defined  as  the  users  or  target  of  the  out¬ 
puts. 

•  Outcomes  (intermediate  or  end)  are  the  changes  that  occur  and  the  benefits  that 
result  from  the  program  activities  and  outputs.  They  involve  changes  in  knowl¬ 
edge,  attitudes,  and  behaviors.  Intermediate  outcomes  reflect  the  customers’ 
response  to  the  program’s  outputs;  end  outcomes  are  the  desired  results  of  the 
program. 

•  Impact  is  a  program’s  contribution  to  a  societal  outcome. 

The  path  from  inputs  to  outcomes  may  not  be  strictly  linear.  Feedback  can  occur 
en  route,  and  various  “external  factors”  (i.e.,  forces,  including  uncertainties,  outside 
the  program,  such  as  the  enactment  of  a  new  legislative  requirement,  development  of 
a  new  technology,  emergence  of  a  new  security  consideration,  or  reconfiguration  of  a 
partner  agency)  may  influence,  positively  or  negatively,  the  extent  to  which  a  program 
can  achieve  outcomes. 

The  logic  model  can  also  be  used  to  identify  program  boundaries  and  responsi¬ 
bilities;  in  particular,  it  can  be  used  to  show  the  range  of  a  program’s  sphere  of  influ¬ 
ence.  Typically,  as  the  model  flows  from  left  to  right  (see  Figure  A.l),  the  extent  of  the 
program’s  control  over  resources,  actors,  and  events  diminishes.  A  program  might  exert 
direct  control  at  the  outset  and  indirect  influence  at  the  conclusion.  Because  multiple 
actors — and  forces — may  contribute  to  any  outcome,  the  logic  model  can  help  estab¬ 
lish  those  outcomes  for  which  the  program  can  reasonably  take  credit  and  be  held 
responsible.  In  particular,  it  can  be  used  to  shed  light  on  the  unique  contributions  of 
the  program  to  the  outcome. 

The  stylized  logic  model — or  “template” — shown  in  Figure  A.l  can  serve  as  a 
starting  point  in  the  development  of  FVAP’s  logic  model(s),  but  it  will  need  to  be 
modified  to  meet  FVAP’s  needs.  Examples  of  potential  modifications  include  explicit 
identification  of  stakeholders,  specification  of  transfer  activities,  delineation  of  custom¬ 
ers  and  outcomes  (intermediate  and  final),  and  incorporation  of  evaluation  processes. 
RAND  teams  have  used  this  template  to  assist  other  Federal  agencies  in  addressing  a 
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Figure  A.1 
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wide  range  of  institutional  and  policy  concerns.7  In  each  case,  the  modelers  identified 
important  divergences  from  the  template  and  incorporated  them  into  the  development 
of  a  program-specific  framework,  but  striking  a  healthy  balance  between  institutional 
detail  and  functionality  can  present  challenges. 

This  template  requires  strict  vertical  alignment.  What  does  that  mean  in  plain 
English?  A  program’s  strategy  and  operations  should  make  sense  as  a  package.  They 
should  “line  up.”  As  indicated  by  the  double  arrows  in  Figure  A.l,  strategic  goals  should 
relate  to  the  program’s  contribution  to  outcomes;  intermediate  goals  should  relate  to 
customer  activities,  which  might  involve  changes  in  knowledge,  attitudes,  or  behavior; 
and  so  forth.  Similarly,  performance  measures  that  are  posited  for  each  stage  should 
enable  programs  to  gauge  progress  in  meeting  the  corresponding  goals  and  objectives. 
Different  institutions  may  adopt  different  vocabularies  to  describe  these  features,  but 
the  principle  of  vertical  alignment  should  hold  regardless  of  nomenclature.  In  any  case, 


7  For  examples,  see  Greenfield,  Williams,  and  Eiseman,  2006;  Williams  et  al.,  2009;  and  Greenfield,  Willis, 
and  LaTourrette,  2012. 
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relationships  between  operations,  goals,  and  measures  should  be  direct,  transparent, 
and  supportable  with  evidence. 

A  primary  strength  of  the  template  is  the  close  integration  of  the  operational  and 
strategic  domains,  thus,  its  capacity  to  serve  multiple  roles  with  a  high  degree  of  inter¬ 
nal  consistency. 

•  As  a  tool  for  fleshing  out  the  connections  and  relationships  between  inputs,  activi¬ 
ties,  outputs,  customers,  and  outcomes,  the  model  can  be  used  to  better  focus  a 
program’s  operations — and  direct  its  resources — to  help  fill  a  societal  need. 

•  As  a  foundation  for  establishing  a  program’s  strategy,  the  model  can  be  used  to 
“walk  back”  from  a  program’s  overarching  mission — its  reason  for  existing — to 
formulate  goals  and  objectives  that  speak  to  its  operations. 

•  As  a  communication  device,  the  model  can  be  used  to  provide  internal  and  exter¬ 
nal  audiences,  including  program  partners,  legislators,  and  other  stakeholders, 
with  a  clear  representation  of  the  program’s  operations  and  intent.  The  model 
can  also  be  used  to  delineate  a  program’s  boundaries  and  responsibilities,  thereby 
clarifying  the  meaning  of  “impact”  and  “results”  as  they  relate  to  the  program. 

•  As  a  tool  for  performance  evaluation,  it  can  also  be  used  to  select  well-aligned 
measures,  i.e.,  measures  that  closely  line  up  with  or  correspond  to  a  program’s 
stated  goals  and  objectives,  and,  thus,  provide  an  appropriate  gauge  of  impact  or 
results. 

Notwithstanding  the  broad  applicability  of  logic  modeling  in  general  and  the 
RAND  template  in  particular,  the  focus  of  this  project,  i.e.,  “FVAP  and  the  Road 
Ahead,”  is  expected  to  be  more  on  operations,  goals,  and  objectives  and  less  on  perfor¬ 
mance  measurement. 
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Agenda 

The  following  agenda  was  prepared  for  the  logic-modeling  workshop,  held  at  FVAP, 
with  FVAP  staff,  including  leadership,  on  May  1-2,  2013.  Although  not  reflected  in  the 
agenda,  the  second  day  concluded  with  a  conversation  about  stakeholder  perceptions. 

May  1,  2013 

Full  Group  Participation 

1:30-1:35  Welcome  (FVAP) 

1:35-1:45  Objectives  and  expectations  (RAND) 

1:45-2:30  Logic  Modeling  101  (RAND-led) 

2:30-2:45  Q&A 
2:45-3:00  Break 

3:00-4:00  Why  Does  FVAP  Exist?  (RAND-led) 

4:00-4:30  Day  2  Preview,  including  breakout  group  assignments 

May  2,  2013 

Consecutive  Breakout  Groups 

8:30-9:25  Breakout  group  I  Voting  Assistance 
9:30-10:25  Breakout  group  II  Election  Official  Assistance 
10:30-11:25  Breakout  group  III  Technology 
11:30-12:25  Breakout  group  IV  Mission  Support3 
a  This  group  included  the  communication  service  group. 

"Who,"  "What,"  and  "Why"  (RAND-Led) 

•  What  inputs,  activities,  and  outputs  are  associated  with  your  program  area? 

•  Flow  do  your  outputs  get  used  and  by  whom? 

•  What  are  the  intended  outcomes? 

•  What  are  the  program  area’s  boundaries? 
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•  What  types  of  external  factors  come  into  play? 

12:30-1:00  Lunch 

Full  Group  Participation 

1:00-2:00  Reconvene,  reflect,  and  discuss  (RAND-led) 

2:00-2:15  Break 

2:15-3:45  Laying  the  foundation  for  FVAP's  logic  model  together  (RAND-led) 

3:45-4:00  Recap  and  next  steps  (RAND  and  FVAP) 


Discussion  Points 

The  following  discussion  points  (verbatim)  were  sent  to  FVAP  as  read-ahead  material 
for  the  logic-modeling  workshop,  held  on  May  1-2,  2013. 

Introduction 

For  the  purposes  of  these  discussion  points,  please  think  of  FVAP  as  a  “program,” 
as  the  acronym  implies,  and  your  work  as  occurring  in  a  “program  area”  within  that 
program.  That  area,  however  you  choose  to  define  it,  could  correspond  to  an  FVAP 
division,  such  as  “election  official  assistance”;  cut  across  divisions,  as  might  be  true  in 
the  case  of  legal  affairs  and  policy;  or  constitute  a  subset  of  a  division.  These  points, 
which  are  intended  to  foster  discussion  today  and  in  the  future,  address  three  basic 
logic-modeling  concepts:  (1)  who  are  “you,”  as  a  program  or  program  area;  (2)  what  are 
“you”  doing;  and  (3)  why  are  “you”  doing  it?  You  might  find  that  some  of  the  points 
seem  to  repeat  or  overlap,  but,  sometimes,  asking  a  question  in  different  ways  or  from 
different  perspectives  can  help  generate  a  better  discussion. 

Program  Area  Definition 

•  How  are  you  defining  your  program  area? 

•  Where  does  it  stand  in  relation  to  other  FVAP  program  areas;  that  is,  in  what 
division  or  divisions  does  your  program  area  reside  and  with  which  other  divi¬ 
sions  and/or  program  areas  does  it  interact?  How  do  your  actions  affect  them; 
how  do  their  actions  affect  you?  Do  you  provide  support  to  others  within  FVAP 
and  do  others  provide  support  to  you?  If  so,  in  what  ways  do  you/they  provide 
support? 

Program/Program  Area  Mission  and  Goals 

•  What  is  your  program/program  area  trying  to  achieve  and  why? 

•  Who  are  its  primary  customers,  partners,  and  other  stakeholders? 

•  How  do  you  define  them? 
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Program/Program  Area  Inputs 

•  What  types  of  physical  and  institutional  infrastructure  support  activities  in  your 
program  area,  e.g.,  IT  systems,  planning  and  funding  processes  .  .  .  ? 

•  What  other  resources  do  you  use  to  undertake  or  plan  for  activities,  e.g.,  funding, 
staff,  customer/partner/stakeholder  feedback,  survey  data,  legal  authority,  plan¬ 
ning  documents,  DoD  instructions  .  .  .  ? 

Program/Program  Area  Activities 

•  What  does  your  program/program  area  do,  e.g.,  analyzes  survey  findings,  devel¬ 
ops  training  tools,  awards  grants,  builds  capacity,  develops  and/or  validates  new 
methods  and  technologies,  assesses  threats,  responds  to  Congressional  inquiries 
.  .  .  ? 

•  Do  you  work  with  partners  inside  FVAP,  inside  DoD,  outside  DoD?  If  so,  how, 
how  often,  and  to  what  end?  Are  they  central  to  your  activities? 

Program/Program  Area  Outputs 

•  What  does  your  program/program  area  “produce,”  e.g.,  reports,  data  bases,  meth¬ 
ods,  tools,  technologies,  training/education  materials,  workshops,  pilot  programs, 
demonstration  projects,  best  practices,  marketing/outreach  materials,  Congres¬ 
sional  replies,  other? 

Dissemination  and  Transfer  of  Outputs 

•  How  are  the  outputs  of  your  program/program  area  disseminated  or  transferred 
to  their  intended  audiences,  e.g.,  do  you  post  materials  on  website,  provide  email 
alerts,  and/or  convene  meetings  .  .  .  ?  Are  strategies  in  place  to  ensure  effective 
dissemination  and  transfer  of  your  outputs?  If  so,  what  are  they? 

•  What  individuals,  agencies,  NGOs,  businesses  and/or  other  entities  receive  or 
request  the  outputs,  both  within  and  outside  of  FVAP,  and  what  are  their  respec¬ 
tive  roles  in  the  voting  community?  (Some  outputs  might  be  intended  to  meet 
the  needs  of  intermediaries,  who  might,  in  turn,  pass  the  outputs  on  to  other 
stakeholders;  other  outputs  might  flow  directly  to  ultimate  beneficiaries,  e.g., 
UOCAVA  voters.)  How  significant  are  these  individuals/entities  in  the  voting 
community? 

Outcomes  and  Program/Program  Area  Contribution 

•  How  do  these  individuals,  agencies,  businesses,  etc.  use — or  respond  to — the 
outputs  that  flow  from  program/program  area  activities?  Do  they  use  the  outputs 
in  their  current  form,  as  produced,  or  do  they  modify  them? 

•  In  what  ways  might  their  use  of  program/program  area  outputs  result  in  changes 
in  knowledge,  attitudes,  and  behavior?  At  what  level  are  these  changes  expected 
to  occur,  e.g.,  individual,  agency,  NGO,  business,  etc.? 
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•  What  is  the  intended  outcome  of  those  changes? 

•  How  will  this  chain  of  events — and  your  program/program  area’s  contribution, 
in  particular — help  to  fill  a  societal  need  or  solve  a  societal  problem? 

•  Who  will  benefit,  ultimately? 

Program/Program  Area  Responsibilities 

•  What  are  the  program/program  area’s  boundaries?  How  do  its  activities  relate  to 
those  of  your  customers,  partners,  and  other  stakeholders? 

•  What  is  the  program/program  area’s  niche  in  the  voting  community? 

External  Factors 

•  What  outside  forces  affect  your  use  of  inputs,  your  activities,  the  outputs  of  your 
activities,  and  the  effects  of  your  activities,  e.g.,  legislative  actions,  changes  in 
technology,  changes  in  structures  of  other  agencies  .  .  .  ? 
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Statutory  Requirements  Pertaining  to  the  Federal  Voting 
Assistance  Program 


This  appendix  lists  the  provisions  in  10  U.S.C.  §  1566  and  §  1566a  and  42  U.S.C. 
§§  1973ff  et  seq.1  and  §  15344  that  could  assign  responsibilities  to  FVAP  as  the  “federal 
voting  assistance  program”  or  the  delegate  of  the  “presidential  designee.”2  In  general, 
the  appendix  does  not  list  provisions  that  name  the  “Secretary  of  Defense,”  “Secretary,” 
or  “Department  of  Defense”  without  specific  reference  to  the  presidential  designee  or 
federal  voting  assistance  program.3  However,  it  includes  the  Electronic  Voting  Dem¬ 
onstration  Project.  The  provision  that  established  the  project  did  not  assign  respon¬ 
sibility  to  either  the  presidential  designee  or  a  federal  voting  assistance  program,  but 
the  project  has  been  associated  closely  with  FVAP  through  appropriations,  policy,  and 
practice.4  The  appendix  also  includes  one  broadly  worded  provision  found  in  42  U.S.C. 
§§  1973gg  et  seq.  that  calls  on  federal,  state,  and  local  governments  to  promote  the 
exercise  of  the  right  to  vote. 

It  is  important  to  note  that  even  the  provisions  that  specifically  assign  responsibil¬ 
ity  to  the  presidential  designee  or  a  federal  voting  assistance  program  need  not  create  a 
requirement  for  FVAP.  For  example,  the  Secretary  of  Defense  could  choose  to  delegate 
responsibility  elsewhere  and  appears  to  have  done  so  in  the  cases  of  some  provisions 
concerning  ballot  collection  and  delivery.  In  those  cases,  responsibility,  as  set  out  in 
DoDI  1000.04,  2012,  rests  with  MPSA.  Moreover,  it  is  sometimes  the  case  that  the 
term  federal  voting  assistance  program  is  used  to  encompass  all  federal  voting  assistance, 
whether  or  not  it  is  tied  to  FVAP,  the  agency. 


1  Following  the  completion  of  the  analysis  for  this  report,  the  U.S.C.  designations  for  voting-related  provisions, 
including  those  in  the  totality  of  §  1973ff  and  §  1973gg,  were  recast  under  their  own  title,  specific  to  voting,  i.e., 
Title  52,  Voting  and  Elections.  For  the  new  code  and  a  crosswalk,  see  52  U.S.C.  and  Office  of  the  Law  Revision 
Counsel,  undated,  respectively. 

2  See  42  U.S.C.  §  1973ff(a)  and  EO  12642  (Reagan,  1988),  both  excerpted  below. 

3  We  do  not,  for  example,  reference  certain  provisions  that  address  regulatory  or  decisionmaking  authority  (see 
10  U.S.C.  §  1566[a]  and  §  1566a[a],  [c],  [e],  and  [f])  or  delivery  of  mail  from  overseas  (see  10  U.S.C.  §  1566[g]). 
Similarly,  we  do  not  include  provisions  found  in  42  U.S.C.  §  1973gg-5,  Voter  Registration  Agencies.  However, 
we  note  that  DoDI  1000.04,  2012,  provided  a  role  for  FVAP  in  the  implementation  of  42  U.S.C.  §  1973gg-5. 

4  Subsequent  to  this  analysis.  Section  593  of  the  National  Defense  Authorization  Act  for  Fiscal  Year  2015,  titled 
“Repeal  of  Electronic  Voting  Demonstration  Project”  eliminated  the  requirement. 
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Provisions  from  U.S.  Code,  Title  10,  Armed  Forces 

§  1566a.  Voting  Assistance:  Voter  Assistance  Offices 

(d)  Outreach.  The  Secretary  of  each  military  department,  or  the  Presidential 
designee,5  shall  take  appropriate  actions  to  inform  absent  uniformed  services  voters  of 
the  assistance  available  under  subsection  (a),  including 

(1)  the  availability  of  information  and  voter  registration  assistance  at  offices 
designated  under  subsection  (a);  and 

(2)  the  time,  location,  and  manner  in  which  an  absent  uniformed  services 
voter  may  utilize  such  assistance. 


Provisions  from  U.S.  Code,  Title  42,  The  Public  Health  and  Welfare 

§  1973ff.  Federal  Responsibilities 

(a)  Presidential  designee.  The  President  shall  designate  the  head  of  an  executive 
department  to  have  primary  responsibility  for  Federal  functions  under  this  subchapter, 
(b)  Duties  of  Presidential  designee.  The  Presidential  designee  shall — 

(1)  consult  State  and  local  election  officials  in  carrying  out  this  subchapter, 
and  ensure  that  such  officials  are  aware  of  the  requirements  of  this  Act; 

(2)  prescribe  an  official  post  card  form,  containing  both  an  absentee  voter 
registration  application  and  an  absentee  ballot  application,  for  use  by  the  States  as 
required  under  section  1973ff-l(a)(4)  of  this  title; 

(3)  carry  out  section  1973ff-2  of  this  title  with  respect  to  the  Federal  write-in 
absentee  ballot  for  absent  uniformed  services  voters  and  overseas  voters  in  general 
elections  for  Federal  office; 

(4)  prescribe  a  suggested  design  for  absentee  ballot  mailing  envelopes; 

(5)  compile  and  distribute 

(A)  descriptive  material  on  State  absentee  registration  and  voting  pro¬ 
cedures,  and 

(B)  to  the  extent  practicable,  facts  relating  to  specific  elections,  includ¬ 
ing  dates,  offices  involved,  and  the  text  of  ballot  questions; 

(6)  not  later  than  the  end  of  each  year  after  a  Presidential  election  year, 
transmit  to  the  President  and  the  Congress  a  report  on  the  effectiveness  of  assis¬ 
tance  under  this  subchapter,  including  a  statistical  analysis  of  uniformed  services 
voter  participation,  a  separate  statistical  analysis  of  overseas  nonmilitary  partici¬ 
pation,  and  a  description  of  State-Federal  cooperation; 


5  Here  and  throughout  this  appendix,  we  have  bolded  all  instances  of  the  phrases  presidential  designee  and  fed- 
eral  voting  assistance  program  to  highlight  the  roles  of  these  actors  in  each  provision. 
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(7)  prescribe  a  standard  oath  for  use  with  any  document  under  this  sub¬ 
chapter  affirming  that  a  material  misstatement  of  fact  in  the  completion  of  such  a 
document  may  constitute  grounds  for  a  conviction  for  perjury; 

(8)  carry  out  section  1973ff-2a  of  this  title  with  respect  to  the  collection  and 
delivery  of  marked  absentee  ballots  of  absent  overseas  uniformed  services  voters 
in  elections  for  Federal  office; 

(9)  to  the  greatest  extent  practicable,  take  such  actions  as  may  be  necessary 

(A)  to  ensure  that  absent  uniformed  services  voters  who  cast  absentee 
ballots  at  locations  or  facilities  under  the  jurisdiction  of  the  Presidential 
designee  are  able  to  do  so  in  a  private  and  independent  manner;  and 

(B)  to  protect  the  privacy  of  the  contents  of  absentee  ballots  cast  by 
absentee  uniformed  services  voters  and  overseas  voters  while  such  ballots  are 
in  the  possession  or  control  of  the  Presidential  designee; 

(10)  carry  out  section  1973ff-2b  of  this  title  with  respect  to  Federal  Voting 
Assistance  Program  Improvements;  and 

(11)  working  with  the  Election  Assistance  Commission  and  the  chief  State 
election  official  of  each  State,  develop  standards 

(A)  for  States  to  report  data  on  the  number  of  absentee  ballots  trans¬ 
mitted  and  received  under  section  1973ff-l(c)  of  this  title  and  such  other 
data  as  the  Presidential  designee  determines  appropriate;  and 

(B)  for  the  Presidential  designee  to  store  the  data  reported. 

(c)  Duties  of  other  Federal  officials 

(1)  In  general.  The  head  of  each  Government  department,  agency,  or  other 
entity  shall,  upon  request  of  the  Presidential  designee,  distribute  balloting  mate¬ 
rials  and  otherwise  cooperate  in  carrying  out  this  subchapter. 

(2)  Administrator  of  General  Services.  As  directed  by  the  Presidential  des¬ 
ignee,  the  Administrator  of  General  Services  shall  furnish  official  post  card  forms 
(prescribed  under  subsection  (b)  of  this  section)  and  Federal  write-in  absentee  bal¬ 
lots  (prescribed  under  section  1973ff-2  of  this  title). 

(d)  Authorization  of  appropriations  for  carrying  out  Federal  Voting  Assistance 
Program  Improvements.  There  are  authorized  to  be  appropriated  to  the  Presidential 
designee  such  sums  as  are  necessary  for  purposes  of  carrying  out  subsection  (b)(10). 

§  1973ff.  Note 

Electronic  Voting  Demonstration  Project.  Pub.  L.  107-107,  div.  A,  title  XVI,  §  1604, 
Dec.  28,  2001,  115  Stat.  1277,  as  amended  by  Pub.  L.  108-375,  div.  A,  title  V,  §  567, 
Oct.  28,  2004,  118  Stat.  1919,  provided  that: 

(a)  Establishment  of  Demonstration  Project. 

(1)  In  general.  Subject  to  paragraph  (2),  the  Secretary  of  Defense  shall  carry 
out  a  demonstration  project  under  which  absent  uniformed  services  voters  are 
permitted  to  cast  ballots  in  the  regularly  scheduled  general  election  for  Federal 
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office  for  November  2002  through  an  electronic  voting  system.  The  project  shall 
be  carried  out  with  participation  of  sufficient  numbers  of  absent  uniformed  ser¬ 
vices  voters  so  that  the  results  are  statistically  relevant. 

(2)  Authority  to  delay  implementation.  If  the  Secretary  of  Defense  deter¬ 
mines  that  the  implementation  of  the  demonstration  project  under  paragraph  (1) 
with  respect  to  the  regularly  scheduled  general  election  for  Federal  office  for 
November  2002  may  adversely  affect  the  national  security  of  the  United  States, 
the  Secretary  may  delay  the  implementation  of  such  demonstration  project  until 
the  first  regularly  scheduled  general  election  for  Federal  office  which  occurs  after 
the  Election  Assistance  Commission  notifies  the  Secretary  that  the  Commission 
has  established  electronic  absentee  voting  guidelines  and  certifies  that  it  will  assist 
the  Secretary  in  carrying  out  the  project.  The  Secretary  shall  notify  the  Com¬ 
mittee  on  Armed  Services  and  the  Committee  on  Rules  and  Administration  of 
the  Senate  and  the  Committee  on  Armed  Services  and  the  Committee  on  House 
Administration  of  the  House  of  Representatives  of  any  decision  to  delay  imple¬ 
mentation  of  the  demonstration  project. 

(b)  Coordination  with  State  Election  Officials.  The  Secretary  shall  carry  out  the 
demonstration  project  under  this  section  through  cooperative  agreements  with  State 
election  officials  of  States  that  agree  to  participate  in  the  project. 

(c)  Report  to  Congress.  Not  later  than  June  1  of  the  year  following  the  year 
in  which  the  demonstration  project  is  conducted  under  this  section,  the  Secretary  of 
Defense  shall  submit  to  Congress  a  report  analyzing  the  demonstration  project.  The 
Secretary  shall  include  in  the  report  any  recommendations  the  Secretary  considers 
appropriate  for  continuing  the  project  on  an  expanded  basis  for  absent  uniformed  ser¬ 
vices  voters  during  the  next  regularly  scheduled  general  election  for  Federal  office. 

(d)  Definitions.  In  this  section: 

(1)  Absent  uniformed  services  voter.  The  term  ‘absent  uniformed  services 
voter’  has  the  meaning  given  that  term  in  section  107(1)  of  the  Uniformed  and 
Overseas  Citizens  Absentee  Voting  Act  (42  U.S.C.  1973ff-6(l)). 

(2)  State.  The  term  ‘State’  includes  the  District  of  Columbia,  the  Common¬ 
wealth  of  Puerto  Rico,  Guam,  the  Virgin  Islands,  and  American  Samoa.” 

[The  notes  accompanying  42  USC  §  1973ff  also  include  a  now-expired  require¬ 
ment  for  “Governors’  Reports  on  Implementation  of  Recommendations  for  Changes 
in  State  Law  Made  under  the  Federal  Voting  Assistance  Program.”] 

Executive  Order  Number  12642.  Designation  of  Secretary  of  Defense  as  Presidential 
Designee 

Ex.  Ord.  No.  12642,  June  8,  1988,  53  F.R.  21975,  provided: 

By  virtue  of  the  authority  vested  in  me  as  President  by  the  Constitution  and 
laws  of  the  United  States  of  America,  including  section  101(a)  of  the  Uniformed  and 
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Overseas  Citizens  Absentee  Voting  Act  (Public  Law  99-410)  (“the  Act”)  [42  U.S.C. 
1973ff(a)],  it  is  hereby  ordered  as  follows: 

Section  1.  The  Secretary  of  Defense  is  hereby  designated  as  the  “Presiden¬ 
tial  designee”  under  Title  I  of  the  Act  [42  U.S.C.  1973ff  et  seq.]. 

Sec.  2.  In  order  to  effectuate  the  purposes  of  the  Act  [see  Short  Title  note 
above],  the  Secretary  of  Defense  is  hereby  authorized  to  delegate  any  or  all  of  the 
functions,  responsibilities,  powers,  authority,  or  discretion  devolving  upon  him 
in  consequence  of  this  Order  to  any  person  or  persons  within  the  Department  of 
Defense. 

§  1973ff-1.  State  Responsibilities 

(a)  In  general.  Each  State  shall 

(5)  if  the  State  requires  an  oath  or  affirmation  to  accompany  any  document 
under  this  subchapter,  use  the  standard  oath  prescribed  by  the  Presidential  des¬ 
ignee  under  section  1973ff(b)(7)  of  this  title; 

(11)  report  data  on  the  number  of  absentee  ballots  transmitted  and  received 
under  subsection  (c)  and  such  other  data  as  the  Presidential  designee  determines 
appropriate  in  accordance  with  the  standards  developed  by  the  Presidential  des¬ 
ignee  under  section  1973ff(b)(ll)  of  this  title. 

(e)(4)  Availability  and  maintenance  of  online  repository  of  State  contact  informa¬ 
tion.  The  Federal  Voting  Assistance  Program  of  the  Department  of  Defense  shall 
maintain  and  make  available  to  the  public  an  online  repository  of  State  contact  infor¬ 
mation  with  respect  to  elections  for  Federal  office,  including  the  single  State  office  des¬ 
ignated  under  subsection  (b)  and  the  means  of  electronic  communication  designated 
under  paragraph  (l),6  to  be  used  by  absent  uniformed  services  voters  and  overseas 
voters  as  a  resource  to  send  voter  registration  applications  and  absentee  ballot  applica¬ 
tions  to  the  appropriate  jurisdiction  in  the  State. 

(g)  Hardship  exemption 

(1)  In  general.  If  the  chief  State  election  official  determines  that  the  State  is 
unable  to  meet  the  requirement  under  subsection  (a)(8)(A)  with  respect  to  an  elec¬ 
tion  for  Federal  office  due  to  an  undue  hardship  described  in  paragraph  (2)(B), 
the  chief  State  election  official  shall  request  that  the  Presidential  designee  grant 
a  waiver  to  the  State  of  the  application  of  such  subsection. 

(2)  Approval  of  waiver  request.  After  consulting  with  the  Attorney  General, 
the  Presidential  designee  shall  approve  a  waiver  request  under  paragraph  (1) 


®  Section  1973ff-l(e)(l)  pertains  to  states’  designation  of  not  less  than  one  means  of  electronic  communication 
for  various  purposes,  including  information  transmission. 
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if  the  Presidential  designee  determines  each  of  the  following  requirements  are 
met: 

(A)  The  comprehensive  plan  under  subparagraph  (D)  of  such  para¬ 
graph  provides  absent  uniformed  services  voters  and  overseas  voters  suffi¬ 
cient  time  to  receive  absentee  ballots  they  have  requested  and  submit  marked 
absentee  ballots  to  the  appropriate  State  election  official  in  time  to  have  that 
ballot  counted  in  the  election  for  Federal  office. 

(B)  One  or  more  of  the  following  issues  creates  an  undue  hardship  for 
the  State: 

(i)  The  State’s  primary  election  date  prohibits  the  State  from  com¬ 
plying  with  subsection  (a)(8)(A). 

(ii)  The  State  has  suffered  a  delay  in  generating  ballots  due  to  a 
legal  contest. 

(iii)  The  State  Constitution  prohibits  the  State  from  complying 
with  such  subsection. 

(3)  Timing  of  waiver 

(A)  In  general.  Except  as  provided  under  subparagraph  (B),  a  State  that 
requests  a  waiver  under  paragraph  (1)  shall  submit  to  the  Presidential  des¬ 
ignee  the  written  waiver  request  not  later  than  90  days  before  the  election 
for  Federal  office  with  respect  to  which  the  request  is  submitted.  The  Presi¬ 
dential  designee  shall  approve  or  deny  the  waiver  request  not  later  than 
65  days  before  such  election. 

(B)  Exception.  If  a  State  requests  a  waiver  under  paragraph  (1)  as  the 
result  of  an  undue  hardship  described  in  paragraph  (2)(B)  (ii),  the  State  shall 
submit  to  the  Presidential  designee  the  written  waiver  request  as  soon  as 
practicable.  The  Presidential  designee  shall  approve  or  deny  the  waiver 
request  not  later  than  5  business  days  after  the  date  on  which  the  request  is 
received. 

(4)  Application  of  waiver.  A  waiver  approved  under  paragraph  (2)  shall  only 
apply  with  respect  to  the  election  for  Federal  office  for  which  the  request  was 
submitted.  For  each  subsequent  election  for  Federal  office,  the  Presidential  des¬ 
ignee  shall  only  approve  a  waiver  if  the  State  has  submitted  a  request  under  para¬ 
graph  (1)  with  respect  to  such  election. 
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§  1 973ff-1 .  Note 

Development  of  Standardized  Format  for  Reports.  Pub.  L.  107-252,  tide  VII,  §  703(b), 
Oct.  29,  2002,  116  Stat.  1724,  provided  that: 

The  Election  Assistance  Commission,  working  with  the  Election  Assistance  Com¬ 
mission  Board  of  Advisors7  and  the  Election  Assistance  Commission  Standards 
Board,  shall  develop  a  standardized  format  for  the  reports  submitted  by  States  and 
units  of  local  government  under  section  102(c)  of  the  Uniformed  and  Overseas 
Citizens  Absentee  Voting  Act  [42  U.S.C.  1973ff-l(c)]  (as  added  by  subsection  (a)), 
and  shall  make  the  format  available  to  the  States  and  units  of  local  government 
submitting  such  reports. 


§  1973ff-2.  Federal  Write-In  Absentee  Ballot  in  General  Elections  for  Federal  Office 
for  Absent  Uniformed  Services  Voters  and  Overseas  Voters 

(a)  In  general 

(1)  Federal  write-in  absentee  ballot.  The  Presidential  designee  shall  pre¬ 
scribe  a  Federal  write-in  absentee  ballot  (including  a  secrecy  envelope  and  mailing 
envelope  for  such  ballot)  for  use  in  general,  special,  primary,  and  runoff  elections 
for  Federal  office  by  absent  uniformed  services  voters  and  overseas  voters  who 
make  timely  application  for,  and  do  not  receive,  States8  absentee  ballots. 

(2)  Promotion  and  expansion  of  use  of  Federal  write-in  absentee  ballots 

(A)  In  general.  Not  later  than  December  31,  2011,  the  Presidential 
designee  shall  adopt  procedures  to  promote  and  expand  the  use  of  the  Fed¬ 
eral  write-in  absentee  ballot  as  a  back-up  measure  to  vote  in  elections  for 
Federal  office. 

(B)  Use  of  technology.  Under  such  procedures,  the  Presidential  desig¬ 
nee  shall  utilize  technology  to  implement  a  system  under  which  the  absent 
uniformed  services  voter  or  overseas  voter  may 

(i)  enter  the  address  of  the  voter  or  other  information  relevant  in 
the  appropriate  jurisdiction  of  the  State,  and  the  system  will  generate 
a  list  of  all  candidates  in  the  election  for  Federal  office  in  that  jurisdic¬ 
tion;  and 

(ii)  submit  the  marked  Federal  write-in  absentee  ballot  by  printing 
the  ballot  (including  complete  instructions  for  submitting  the  marked 
Federal  write-in  absentee  ballot  to  the  appropriate  State  election  official 
and  the  mailing  address  of  the  single  State  office  designated  under  sec¬ 
tion  1973ff-l(b)  of  this  title). 


7  The  director  of  FVAP  serves  as  a  member  of  this  board  under  42  U.S.C.  §  15344. 

8  The  U.S.  Code  contains  the  following  endnote:  “So  in  original.  Probably  should  be  ‘States.’” 
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(C)  Authorization  of  appropriations.  There  are  authorized  to  be  appro¬ 
priated  to  the  Presidential  designee  such  sums  as  may  be  necessary  to  carry 
out  this  paragraph. 

(d)  Second  ballot  submission;  instruction  to  absent  uniformed  services  voter  or 
overseas  voter.  An  absent  uniformed  services  voter  or  overseas  voter  who  submits  a 
Federal  write-in  absentee  ballot  and  later  receives  a  State  absentee  ballot,  may  submit 
the  State  absentee  ballot.  The  Presidential  designee  shall  assure  that  the  instructions 
for  each  Federal  write-in  absentee  ballot  clearly  state  that  an  absent  uniformed  services 
voter  or  overseas  voter  who  submits  a  Federal  write-in  absentee  ballot  and  later  receives 
and  submits  a  State  absentee  ballot  should  make  every  reasonable  effort  to  inform  the 
appropriate  State  election  official  that  the  voter  has  submitted  more  than  one  ballot. 

(e)  Use  of  approved  State  absentee  ballot  in  place  of  Federal  write-in  absentee 
ballot.  The  Federal  write-in  absentee  ballot  shall  not  be  valid  for  use  in  a  general,  spe¬ 
cial,  primary,  or  runoff  election  for  Federal  office  if  the  State  involved  provides  a  State 
absentee  ballot  that 

(1)  at  the  request  of  the  State,  is  approved  by  the  Presidential  designee  for 
use  in  place  of  the  Federal  write-in  absentee  ballot;  and 

(2)  is  made  available  to  absent  uniformed  services  voters  and  overseas  voters 
at  least  60  days  before  the  deadline  for  receipt  of  the  State  ballot  under  State  law. 

§  1973ff-2a.  Procedures  for  Collection  and  Delivery  of  Marked  Absentee  Ballots  of 
Absent  Overseas  Uniformed  Services  Voters 

(a)  Establishment  of  procedures.  The  Presidential  designee  shall  establish  procedures 
for  collecting  marked  absentee  ballots  of  absent  overseas  uniformed  services  voters 
in  regularly  scheduled  general  elections  for  Federal  office,  including  absentee  bal¬ 
lots  prepared  by  States  and  the  Federal  write-in  absentee  ballot  prescribed  under  sec¬ 
tion  1973ff-2  of  this  title,  and  for  delivering  such  marked  absentee  ballots  to  the  appro¬ 
priate  election  officials. 

(b)  Delivery  to  appropriate  election  officials 

(1)  In  general.  Under  the  procedures  established  under  this  section,  the 
Presidential  designee  shall  implement  procedures  that  facilitate  the  delivery  of 
marked  absentee  ballots  of  absent  overseas  uniformed  services  voters  for  regularly 
scheduled  general  elections  for  Federal  office  to  the  appropriate  election  officials, 
in  accordance  with  this  section,  not  later  than  the  date  by  which  an  absentee 
ballot  must  be  received  in  order  to  be  counted  in  the  election. 

(2)  Cooperation  and  coordination  with  the  United  States  Postal  Service. 
The  Presidential  designee  shall  carry  out  this  section  in  cooperation  and  coor¬ 
dination  with  the  United  States  Postal  Service,  and  shall  provide  expedited  mail 
delivery  service  for  all  such  marked  absentee  ballots  of  absent  uniformed  services 
voters  that  are  collected  on  or  before  the  deadline  described  in  paragraph  (3)  and 
then  transferred  to  the  United  States  Postal  Service. 
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(3)  Deadline  described 

(A)  In  general.  Except  as  provided  in  subparagraph  (B),  the  deadline 
described  in  this  paragraph  is  noon  (in  the  location  in  which  the  ballot  is 
collected)  on  the  seventh  day  preceding  the  date  of  the  regularly  scheduled 
general  election  for  Federal  office. 

(B)  Authority  to  establish  alternative  deadline  for  certain  locations.  If 
the  Presidential  designee  determines  that  the  deadline  described  in  sub- 
paragraph  (A)  is  not  sufficient  to  ensure  timely  delivery  of  the  ballot  under 
paragraph  (1)  with  respect  to  a  particular  location  because  of  remoteness 
or  other  factors,  the  Presidential  designee  may  establish  as  an  alternative 
deadline  for  that  location  the  latest  date  occurring  prior  to  the  deadline 
described  in  subparagraph  (A)  which  is  sufficient  to  provide  timely  delivery 
of  the  ballot  under  paragraph  (1). 

(4)  No  postage  requirement.  In  accordance  with  section  3406  of  title  39, 
such  marked  absentee  ballots  and  other  balloting  materials  shall  be  carried  free 
of  postage. 

(5)  Date  of  mailing.  Such  marked  absentee  ballots  shall  be  postmarked  with 
a  record  of  the  date  on  which  the  ballot  is  mailed. 

(c)  Outreach  for  absent  overseas  uniformed  services  voters  on  procedures.  The 
Presidential  designee  shall  take  appropriate  actions  to  inform  individuals  who  are 
anticipated  to  be  absent  overseas  uniformed  services  voters  in  a  regularly  scheduled 
general  election  for  Federal  office  to  which  this  section  applies  of  the  procedures  for  the 
collection  and  delivery  of  marked  absentee  ballots  established  pursuant  to  this  section, 
including  the  manner  in  which  such  voters  may  utilize  such  procedures  for  the  submit¬ 
tal  of  marked  absentee  ballots  pursuant  to  this  section. 

(e)  Authorization  of  appropriations.  There  are  authorized  to  be  appropriated  to  the 
Presidential  designee  such  sums  as  may  be  necessary  to  carry  out  this  section. 

§  1973ff-2b.  Federal  Voting  Assistance  Program  Improvements 

(a)  Duties.  The  Presidential  designee  shall  carry  out  the  following  duties: 

(1)  Develop  online  portals  of  information  to  inform  absent  uniformed  ser¬ 
vices  voters  regarding  voter  registration  procedures  and  absentee  ballot  proce¬ 
dures  to  be  used  by  such  voters  with  respect  to  elections  for  Federal  office. 

(2)  Establish  a  program  to  notify  absent  uniformed  services  voters  of  voter 
registration  information  and  resources,  the  availability  of  the  Federal  postcard 
application,  and  the  availability  of  the  Federal  write-in  absentee  ballot  on  the 
military  Global  Network,  and  shall  use  the  military  Global  Network  to  notify 
absent  uniformed  services  voters  of  the  foregoing  90,  60,  and  30  days  prior  to 
each  election  for  Federal  office. 

(b)  Clarification  regarding  other  duties  and  obligations.  Nothing  in  this  section 
shall  relieve  the  Presidential  designee  of  their  duties  and  obligations  under  any  direc- 
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tives  or  regulations  issued  by  the  Department  of  Defense,  including  the  Department  of 
Defense  Directive  1000.04  (or  any  successor  directive  or  regulation)  that  is  not  incon¬ 
sistent  or  contradictory  to  the  provisions  of  this  section. 

(c)  Authorization  of  appropriations.  There  are  authorized  to  be  appropriated  to 
the  Federal  Voting  Assistance  Program  of  the  Department  of  Defense  (or  a  successor 
program)  such  sums  as  are  necessary  for  purposes  of  carrying  out  this  section. 

§  1973ff-4a.  Reporting  Requirements 

(b)  Annual  report  on  effectiveness  of  activities  and  utilization  of  certain  procedures.9 
Not  later  than  March  31  of  each  year,  the  Presidential  designee  shall  transmit  to  the 
President  and  to  the  relevant  committees  of  Congress  a  report  containing  the  follow¬ 
ing  information: 

(1)  An  assessment  of  the  effectiveness  of  activities  carried  out  under  sec¬ 
tion  1973ff-2b  of  this  title,  including  the  activities  and  actions  of  the  Federal 
Voting  Assistance  Program  of  the  Department  of  Defense,  a  separate  assess¬ 
ment  of  voter  registration  and  participation  by  absent  uniformed  services  voters,  a 
separate  assessment  of  voter  registration  and  participation  by  overseas  voters  who 
are  not  members  of  the  uniformed  services,  and  a  description  of  the  cooperation 
between  States  and  the  Federal  Government  in  carrying  out  such  section. 

(2)  A  description  of  the  utilization  of  voter  registration  assistance  under  sec¬ 
tion  1566a  of  title  10,  which  shall  include  the  following: 

(A)  A  description  of  the  specific  programs  implemented  by  each  mili¬ 
tary  department  of  the  Armed  Forces  pursuant  to  such  section. 

(B)  The  number  of  absent  uniformed  services  voters  who  utilized  voter 
registration  assistance  provided  under  such  section. 

(3)  In  the  case  of  a  report  submitted  under  this  subsection  in  the  year  fol¬ 
lowing  a  year  in  which  a  regularly  scheduled  general  election  for  Federal  office 
is  held,  a  description  of  the  utilization  of  the  procedures  for  the  collection  and 
delivery  of  marked  absentee  ballots  established  pursuant  to  section  1973ff-2a  of 
this  title,  which  shall  include  the  number  of  marked  absentee  ballots  collected 
and  delivered  under  such  procedures  and  the  number  of  such  ballots  which  were 
not  delivered  by  the  time  of  the  closing  of  the  polls  on  the  date  of  the  election  (and 
the  reasons  such  ballots  were  not  so  delivered). 

§  1973ff-6.  Definitions 

As  used  in  this  subchapter,  the  term 

(2)  “balloting  materials”  means  official  post  card  forms  (prescribed  under 
section  1973ff  of  this  title),  Federal  write-in  absentee  ballots  (prescribed  under 


9  Section  §  1973ff-4a(a),  which  we  have  omitted,  called  for  a  report  on  status  of  implementation  and  assessment 
of  programs.  This  was  a  one-time  reporting  requirement,  expiring  180  days  after  October  28,  2009. 
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section  1973ff-2  of  this  title),  and  any  State  balloting  materials  that,  as  determined 
by  the  Presidential  designee,  are  essential  to  the  carrying  out  of  this  subchapter; 

§  1973ff-7.  Technology  Pilot  Program 

(a)  Definitions.  In  this  section: 

(1)  Absent  uniformed  services  voter.  The  term  “absent  uniformed  services 
voter”  has  the  meaning  given  such  term  in  section  107(1)  of  the  Uniformed  and 
Overseas  Citizens  Absentee  Voting  Act  (42  U.S.C.  1973ff-6(l)). 

(2)  Overseas  voter.  The  term  “overseas  voter”  has  the  meaning  given  such 
term  in  section  107(5)  of  such  Act  [42  U.S.C.  1973ff-6(5)]. 

(3)  Presidential  designee.  The  term  “Presidential  designee”  means  the  indi¬ 
vidual  designated  under  section  101(a)  of  such  Act  [42  U.S.C.  1973ff(a)]. 

(b)  Establishment 

(1)  In  general.  The  Presidential  designee  may  establish  1  or  more  pilot  pro¬ 
grams  under  which  the  feasibility  of  new  election  technology  is  tested  for  the  ben¬ 
efit  of  absent  uniformed  services  voters  and  overseas  voters  claiming  rights  under 
the  Uniformed  and  Overseas  Citizens  Absentee  Voting  Act  (42  U.S.C.  1973ff  et 
seq.). 

(2)  Design  and  conduct.  The  design  and  conduct  of  a  pilot  program  estab¬ 
lished  under  this  subsection 

(A)  shall  be  at  the  discretion  of  the  Presidential  designee;  and 

(B)  shall  not  conflict  with  or  substitute  for  existing  laws,  regulations, 

or  procedures  with  respect  to  the  participation  of  absent  uniformed  services 

voters  and  military  voters  in  elections  for  Federal  office. 

(c)  Considerations.  In  conducting  a  pilot  program  established  under  subsec¬ 
tion  (b),  the  Presidential  designee  may  consider  the  following  issues: 

(1)  The  transmission  of  electronic  voting  material  across  military  networks. 

(2)  Virtual  private  networks,  cryptographic  voting  systems,  centrally  con¬ 
trolled  voting  stations,  and  other  information  security  techniques. 

(3)  The  transmission  of  ballot  representations  and  scanned  pictures  in  a 
secure  manner. 

(4)  Capturing,  retaining,  and  comparing  electronic  and  physical  ballot 
representations. 

(5)  Utilization  of  voting  stations  at  military  bases. 

(6)  Document  delivery  and  upload  systems. 

(7)  The  functional  effectiveness  of  the  application  or  adoption  of  the  pilot 
program  to  operational  environments,  taking  into  account  environmental  and 
logistical  obstacles  and  State  procedures. 


136  The  Federal  Voting  Assistance  Program  and  the  Road  Ahead 


(d)  Reports.  The  Presidential  designee  shall  submit  to  Congress  reports  on  the 
progress  and  outcomes  of  any  pilot  program  conducted  under  this  subsection,  together 
with  recommendations 

(1)  for  the  conduct  of  additional  pilot  programs  under  this  section;  and 

(2)  for  such  legislation  and  administrative  action  as  the  Presidential  desig¬ 
nee  determines  appropriate. 

(e)  Technical  assistance 

(1)  In  general.  The  Election  Assistance  Commission  and  the  National  Insti¬ 
tute  of  Standards  and  Technology  shall  provide  the  Presidential  designee  with 
best  practices  or  standards  in  accordance  with  electronic  absentee  voting  guide¬ 
lines  established  under  the  first  sentence  of  section  1604(a)(2)  of  the  National 
Defense  Authorization  Act  for  Fiscal  Year  2002  (Public  Law  107-107;  115  Stat. 
1277;  42  U.S.C.  1973ff  note),  as  amended  by  section  567  of  the  Ronald  W.  Reagan 
National  Defense  Authorization  Act  for  Fiscal  Year  2005  (Public  Law  108-375; 
118  Stat.  1919)  to  support  the  pilot  program  or  programs. 

(2)  Report.  In  the  case  in  which  the  Election  Assistance  Commission  has 
not  established  electronic  absentee  voting  guidelines  under  such  section  1604(a) 
(2),  as  so  amended,  by  not  later  than  180  days  after  October  28,  2009,  the  Elec¬ 
tion  Assistance  Commission  shall  submit  to  the  relevant  committees  of  Congress 
a  report  containing  the  following  information: 

(A)  The  reasons  such  guidelines  have  not  been  established  as  of  such 

date. 

(B)  A  detailed  timeline  for  the  establishment  of  such  guidelines. 

(C)  A  detailed  explanation  of  the  Commission’s  actions  in  establishing 

such  guidelines  since  October  28,  2004. 

§  1973gg.  Findings  and  Purposes 

(a)  Findings.  The  Congress  finds  that 

(1)  the  right  of  citizens  of  the  United  States  to  vote  is  a  fundamental  right; 

(2)  it  is  the  duty  of  the  Federal,  State,  and  local  governments  to  promote  the 
exercise  of  that  right;  and 

(3)  discriminatory  and  unfair  registration  laws  and  procedures  can  have  a 
direct  and  damaging  effect  on  voter  participation  in  elections  for  Federal  office 
and  disproportionately  harm  voter  participation  by  various  groups,  including 
racial  minorities. 

§  15344.  Membership  of  Board  of  Advisors 

(a)  In  general.  The  Board  of  Advisors  shall  be  composed  of  37  members  appointed  as 
follows: 

(14)  The  director  of  the  Federal  Voting  Assistance  Program  of  the  Depart¬ 
ment  of  Defense. 
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Survey  Response  Rates  and  Regularly  Scheduled  Reports  to 
Congress 


Response  rates  for  each  of  the  2010  postelection  surveys  are  shown  in  Table  C.l. 
Response  rates  for  ADM  personnel,  ADM  spouses,  overseas  citizens,1  and  UVAOs 
were  low  in  that  year  and  have  been  subject  to  criticism  previously  (see,  for  example, 
GAO,  2006).  ADM  spouses  were  surveyed  for  the  first  time  in  2010  and  again  in  2012. 

The  apparent  burdensomeness  of  the  surveys — in  terms  of  both  length  and 
frequency — might  help  to  explain  the  low  response  rates.  In  1996,  the  response  rate  to 
the  ADM  personnel  survey,  which  asked  45  questions,  was  almost  three  times  higher 
in  than  that  in  2010, 2  when  it  asked  89  questions.  (Figure  C.l  shows  the  number  of 
questions  for  each  survey  year,  starting  in  1996.)  Moreover,  in  recent  years,  the  survey 

Table  C.l 

Survey  Mode,  Sample  Size,  and  Response  Rates  for  2010  Postelection  Surveys 


Population 

Mode  of  Administration 

Approximate 
Sample  Size 

Weighted  Response  Rate 
(%) 

ADM 

Web  only 

73,244 

15 

ADM  spouses 

Web  and  paper 

50,132 

14 

Overseas  citizens 

Web  only  paper  [sic]a 

47,879 

5 

DoD  UVAOs 

Web  only 

9,907 

20 

Department  of  State  VAOs 

Web  only 

238 

90 

Local  election  officials 

Web  and  paper 

7,296 

53 

SOURCE:  FVAP,  20111 
a  As  described  in  the  report  to  Congress. 


1  According  to  FVAP,  the  overseas-citizen  surveys  have  been  discontinued  for  methodological  reasons  and  until 
such  time  as  FVAP  can  reasonably  identify  the  overseas-citizen  voter  population. 

2  The  number  of  questions  declined  from  89  in  2010  to  79  in  2012;  both  surveys  covered  a  wide  range  of  topics, 
including  attitudes  toward  voting,  interest  in  elections,  the  absentee  ballot-request  process,  satisfaction  with  the 
absentee  voting  process,  satisfaction  with  UVAOs  and  IVAOs,  and  satisfaction  with  the  Voting  Assistance  Guide. 
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Figure  C.1 

Numbers  of  Questions  in  Active-Duty  Military  Personnel  Surveys 


Survey  year 

SOURCE:  RAND  staff  analysis  based  on  FVAP,  2013b,  201  If,  201  Id,  2005b,  2001,  1997. 
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has  been  administered  biennially;  prior  to  2010 — and  the  enactment  of  the  MOVE 
Act — the  survey  was  administered  quadrennially. 

FVAP’s  website  contains  reports  to  Congress,  including  the  postelection  survey 
reports  to  Congress,  the  Annual  Report  on  the  Effectiveness  of  Activities  and  Utiliza¬ 
tion  of  Certain  Procedures,  and  various  thematic  reports  on  such  topics  as  barriers  to 
voting  and  the  feasibility  of  technology  solutions.  Here,  we  summarize  the  content  of 
the  postelection  and  annual  reports  and  direct  the  reader  to  the  FVAP  website  for  addi¬ 
tional  information  about  the  others  (see  FVAP,  undated  [c]): 


•  the  postelection  survey  reports  to  Congress,  consisting  of  the  reports  for  the 
years  1996,  2000,  2004,  2008,  2010,  and  2012  (FVAP,  1997,  2001,  2005b, 
2011d,  201  If,  2013b).  The  report  for  2012  describes  survey  results,  including 
registration  and  participation  rates  for  uniformed-service  voters,  collection  and 
delivery  of  ballots  for  overseas  uniformed-service  voters,  FVAP  activities,  federal 
and  state  cooperation,  and  military  and  Department  of  State  voting  assistance. 

•  the  Annual  Report  on  the  Effectiveness  of  Activities  and  Utilization  of  Cer¬ 
tain  Procedures,  consisting  of  reports  for  the  years  2011  (FVAP,  2012)  and 
2009  (FVAP,  2011b).  These  reports  describe  FVAP  activities  and  federal  and 
state  cooperation.  In  years  with  regularly  scheduled  federal  elections,  the  post- 
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election  survey  reports  include  the  descriptions.  In  other  years,  the  annual  reports 
stand  alone  and  include  information  on  special  and  general  elections.3 


The  2011  report  does  not  appear  to  address  the  utilization  of  voter  assistance  under  10  U.S.C.  §  1566a. 


APPENDIX  D 


Project-Management  Tools 


As  we  discussed  in  Chapter  Six,  FVAP  engages  with  its  operating  environment,1  in 
part,  by  building  partnerships  and  conducting  research  and  other  activities.  We  also 
identified  opportunities  for  improving  how  FVAP  goes  about  this  engagement.  In  par¬ 
ticular,  we  examined  FVAP’s  approach  to  developing  and  conducting  research  projects 
and  to  using  the  information  that  that  research  yields,  then  identified  lessons  for  FVAP 
to  enhance  the  use  of  its  analytic  capabilities. 

To  codify  these  lessons,  we  worked  with  FVAP  to  develop  a  set  of  tools  to  help 
guide  FVAP’s  research-related  activities — and  other  projects,  more  generally — and  to 
improve  the  link  between  these  and  FVAP’s  mission,  priorities,  and  operations.  Spe¬ 
cifically,  we  developed  a  checklist  and  worksheet,  derived  from  research  and  program- 
management  best  practices,  and  tailored  to  FVAP’s  particular  needs.  Our  intent  was  to 
help  FVAP  and  its  staff  do  the  following: 

•  Identify  promising  projects. 

•  Strengthen  the  rationale  for  undertaking  projects. 

•  Engage  with  stakeholders  more  transparently. 

The  checklist,  a  sequential  “yes-no”  tool  for  systematic  decisionmaking,  and  the 
worksheet,  an  analytical  tool  to  support  that  decisionmaking,  can  be  used  to  decide 
whether  to  proceed  with  a  new  project,  revise  existing  projects,  or  revise  ongoing 
FVAP  activities,  such  as  training.  (This  appendix  refers  to  those  endeavors,  collectively, 
as  projects.)  The  checklist  and  worksheet  should  be  understood  as  a  framework  for 
approaching  and  thinking  about  projects,  not  as  a  prescribed  formula,  consisting  of 
mandatory,  rote  tasks. 


1  In  Chapter  One,  we  defined  operating  environment  as  including  customers,  partners,  and  other  stakeholders; 
related  organizations;  and  supporting  infrastructure  and  technology. 
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In  applying  the  tools,  F  VAP  would  need  to  consider  whether  the  following  condi¬ 
tions  are  met: 

•  The  project  will  meet  a  need. 

•  The  approach  to  meeting  the  need  is  appropriate. 

•  Stakeholders  will  value  the  project. 

•  The  benefits  of  the  project  justify  the  costs  and  risks. 

•  FVAP  is  the  best  agency  to  perform  the  work. 

•  FVAP  will  know  whether  the  project  is  succeeding. 

•  The  project  is  sustainable. 

•  FVAP  has  a  plan  for  managing  the  project  to  completion  and  beyond. 

Figure  D.l  illustrates  the  relationship  between  the  tools  and  decisionmaking  about 
projects,  both  before  and  after  they  start,  with  three  generic  project  phases:  (1)  analysis 
of  candidate  projects,  (2)  project  implantation  and  management,  and  (3)  transfer,  i.e., 
distribution  and  use,  of  the  results  of  the  completed  project  to  affect  outcomes.  The 
checklist  and  worksheet  are  most  relevant  to  the  first  phase,  involving  analysis  of  the 
candidate  project,  but  they  might  have  additional  downstream  uses.  For  example,  ele¬ 
ments  of  the  worksheet  speak  to  the  development  of  indicators  and  criteria  to  monitor 
and  gauge  the  project’s  success  and  to  the  development  of  a  narrative,  describing  the 
relationship  between  the  project,  its  outputs,  and  desired  outcomes,  which  can  be  used 
to  guide  the  dissemination  and  use  of  project  results. 

Figure  D.l 

Relationship  Between  Tools,  Engagement  Decisions,  and  Project  Life 
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Methodology 

We  developed  the  checklist  and  worksheet  by  reviewing  related  work  in  other  forums, 
including  project-management  tools  that  other  U.S.  government  agencies  have  adopted. 
We  identified  aspects  of  other  tools  that,  in  our  judgment,  are  most  relevant  to  FVAP’s 
needs  and  adapted  them  to  create  a  customized  tool.  At  the  end  of  this  appendix,  we 
describe  the  related  work  and  how  it  informed  our  efforts. 

We  tested  our  tools  by  retrospectively  applying  them  to  three  types  of  prior  FVAP 
projects,  consisting  of  specific  (basic)  research  (e.g.,  the  Overseas  Citizen  Count);  data 
collection  (e.g.,  surveys);  and  grants  (e.g.,  the  Electronic  Absentee  Systems  for  Elec¬ 
tions  1  and  2  grant  programs).  In  addition,  we  applied  elements  of  the  tools  outside 
those  domains  (i.e.,  to  training  activities),  both  to  establish  their  merit  in  relation  to 
other  types  of  projects  and  to  draw  out  lessons  for  training. 


Use  of  the  Checklist  and  Worksheet 

The  checklist  and  worksheet  have  different  target  audiences  and  purposes.  The  check¬ 
list  is  a  tool  for  management  (i.e.,  the  director  or  a  designee)  and  decisionmaking;  the 
worksheet  is  a  tool  for  staff  analysis.  The  checklist  consists  of  a  set  of  yes-no  ques¬ 
tions  that  support  three  overarching,  sequential  decisions  pertaining  to  potential  via¬ 
bility,  readiness,  and  eventual  engagement:  the  decisions  to  seriously  consider  a  proj¬ 
ect,  to  deem  FVAP  equipped  to  engage  in  a  project,  and  to  embark  on  the  project 
(Figure  D.2).2  The  last  decision  (that  of  actual  engagement)  requires  consideration  of 
the  ft  of  the  project  in  the  context  of  the  larger  portfolio  of  FVAP  activities  and  of 
FVAP’s  ability  to  do  the  project  well. 

If  the  director  cannot  say  “yes”  to  potential  viability,  he  or  she  need  not  proceed 
to  readiness;  similarly,  if  he  or  she  cannot  say  “yes”  to  readiness,  the  process  can  end  at 
that  point,  without  further  contemplation  of  actual  engagement. 

Regarding  fit,  a  given  project  might  not  be  as  well-aligned  with  the  agency’s  pri¬ 
orities  as  another,  and  other,  equally  well-aligned  projects  might  yield  greater  benefits 
and,  thus,  represent  a  better  use  of  limited  resources.  Or,  other  projects  could  require 
taking  less  risk.  A  project  might,  for  example,  draw  too  heavily  on  FVAP’s  core  capa¬ 
bilities,  such  that  undertaking  it  would  draw  down  FVAP  capacity  and  jeopardize 
other  FVAP  activities. 

Presumably,  in  addressing  the  first  two  decisions,  regarding  potential  viability 
and  readiness,  FVAP  staff  would  have  presented  some  evidence  of  ability;  however,  we 


2  Figure  D.l  calls  out  the  decision  to  engage.  The  other  two  decisions  are  precursors  that  aim  to  identify  promis¬ 
ing  projects  quickly  and  filter  out  problematic  ones. 
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Figure  D.2 
The  Checklist 


Decision 


Questions 


Should  FVAP 
consider  the 
project? 

1.  Will  the  project  meet  a  need? 

2.  Is  the  approach  fit  for  its  purpose? 

3.  Will  the  right  stakeholders  care  for  the  right  reasons? 

4.  Do  the  benefits  merit  the  costs  and  risks  to  FVAP  and  the 
voting  assistance  system? 

5.  Is  FVAP  the  best  agency  to  engage  in  the  project  in  this  way? 

Is  FVAP  ready 
to  engage  in 
the  project? 

6.  Will  FVAP  know  whether  the  project  is  succeeding? 

7.  Is  the  project  sustainable? 

8.  Does  FVAP  have  a  plan  for  managing  the  project  through  and 
beyond  completion? 

Yes  on  all  = 
promising  project 


Should  FVAP 
engage? 


How  does  the  project  stack  up  against  others  (i.e.,  could  FVAP 
be  doing  something  better  with  its  and  others'  resources)? 

Is  it  something  FVAP  can  do  well? 
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suggest  a  second,  explicit  examination.3  When  a  project  uses  FVAP’s  core  capabilities, 
that  project  might  have  a  better  chance  of  success  because  it  draws  on  FVAP’s  accu¬ 
mulated  knowledge  and  skills.  However,  the  intent  of  some  projects  might  be  to  build 
FVAP  capabilities  or  increase  existing  capacities  around  certain  capabilities.  Thus,  not 
every  project  need  demonstrate  that  it  is  using  FVAP’s  core  capabilities.  In  such  cases, 
FVAP  should  understand  clearly  and  plan  for  how  a  new  capability  or  increased  capac¬ 
ity  will  support  other  projects.  Without  such  an  understanding  and  plan,  FVAP  would 
risk  stranding  the  investment  it  makes  in  capability  or  capacity  by  never  using  it  to 
achieve  outcomes  related  to  mission. 

The  worksheet  sets  out  the  different  types  of  analysis  that  could  be  necessary  to 
inform  the  director’s  decisions,  but  the  level  of  effort  invested  in  the  analysis  should  be 
commensurate  with  the  nature  of  the  project  under  consideration  (Figure  D.3).  Ana¬ 
lytic  effort  takes  time  and  uses  FVAP’s — and  potentially  others’ — resources.  Here  we 
use  size  as  a  proxy  for  all  of  a  project’s  characteristics,  including  the  resources  required 
to  undertake  the  project  and  the  importance  of  the  project  to  FVAP,  stakeholders,  and 
the  voting  assistance  system.  On  that  basis,  large  projects  might  demand  more  atten¬ 
tion  than  medium-sized  projects,  which,  in  turn,  might  demand  more  attention  than 


3  We  differentiate  between  capability  and  capacity.  Capability  is  the  ability  needed  to  do  something,  and  capac¬ 
ity  refers  to  bandwidth.  It  is  possible  to  have  the  capability  but  not  the  capacity  to  do  something. 
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Figure  D.3 

Level  of  Effort  Commensurate  with  Project  Size 


Small  projects  should 
not  require  detailed 
analysis 


Medium-sized  projects 
might  need  more  analysis 
overall  and  of  specific  issues 
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Large  projects 
warrant  detailed 
analysis 


small  projects  would.  If  size  is  a  proxy  for  all  characteristics  of  interest,  the  level  of 
effort  might  not  be  correlated  strictly  to  anticipated  outlays.  For  example,  a  project  that 
is  particularly  high  profile  (e.g.,  of  concern  to  customers,  partners,  or  other  stakehold¬ 
ers)  might  warrant  a  larger  analytic  effort  and  more  deliberation  than  one  that  speaks 
primarily  to  an  F  VAP-internal  audience,  even  if  less  or  equally  costly.  As  noted  at  the 
outset,  the  checklist  and  worksheet  should  foster  a  way  of  approaching  and  thinking 
about  projects  but  should  not  be  construed  as  formulaic,  lock-step  project-management 
tools. 

We  envision  the  process  of  posing  and  answering  the  first  eight  questions  to  be 
iterative.  The  director  might  ask  staff  to  focus  more  attention  on  some  elements  of  the 
worksheet  than  on  others  and,  when  presented  with  the  analysis,  ask  staff  to  dig  more 
deeply  into  certain  issues  or  to  address  perceived  gaps.  In  effect,  the  analysis  might 
prompt  additional  analysis.  The  first  attempt  to  inform  an  answer  need  not  be  the  last. 
An  iterative  process  can  also  strengthen  the  formulation  of  a  project  and  its  implemen¬ 
tation  plan. 

We  now  turn  to  the  elements  of  the  worksheet,  each  of  which  supports  a  checklist 
question,  and  conclude  with  a  discussion  of  the  ultimate  decision  to  engage. 


Elements  of  the  Worksheet 

In  this  section,  we  discuss  and  present  the  elements  of  the  worksheet,  which  contain 
analytical  guidance  to  support  the  first  eight  questions  in  the  checklist  (the  last  two 
questions  are  intended  for  the  director,  not  the  staff). 
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The  analytical  guidance  is  tiered,  in  that  it  starts  with  a  high-level  instruction 
to  identify  a  concept  (e.g.,  characterize  the  needs  a  project  will  meet)  and,  in  most 
instances,  drills  down  to  flesh  out  the  content  of  a  complete  analysis.  In  large  part,  the 
drill-downs  define  the  higher-level  concepts,  either  directly  or  through  examples,  and 
serve  as  documentation. 

Here,  we  offer  some  supplementary  notes  for  potential  users. 

As  noted  above,  not  all  projects  require  complete  analyses  for  all  elements  of  the 
worksheet.  In  some  instances,  a  rough  sketch  might  suffice,  at  least  initially;  in  others, 
not.  In  all  cases,  we  would  expect  staff  to  articulate  a  need,  consisting  of  an  unan¬ 
swered  question,  an  undeveloped  capability,  or  an  insufficient  capacity — with  sup¬ 
porting  evidence — and  to  draw  out  a  plausible  path  from  project  completion  to  the 
attainment  of  an  outcome  in  support  of  FVAP’s  mission,  its  legal  requirements,  or  an 
improvement  of  its  operations.4 

FVAP  staff  should  also  be  able  to  articulate  the  approach — that  is,  the  means  by 
which  the  project  will  meet  the  need  and  achieve  the  intended  outcomes.  In  developing 
the  approach,  we  would  expect  FVAP  staff  to  identify  related  work,  undertaken  previ¬ 
ously  or  currently  under  way  elsewhere,  and  to  discuss  its  implications  for  the  proposed 
project.  If  similar  projects  have  failed,  why  did  they  fail,  and  what  will  be  different 
about  this  project? 

The  worksheet  emphasizes  stakeholders,  both  as  customers  and  partners  with 
wide-ranging  and  potentially  conflicting  interests  and  as  potential  sources  of  expert 
opinion.5  In  considering  and  weighing  stakeholders’  interests  from  the  outset,  FVAP 
might  be  better  positioned  to  meet  their  needs  and  to  forestall  conflict.  To  anticipate 
possible  responses,  FVAP  staff  might  consider  how  a  stakeholder  might  use  or  share 
information  about  the  project  and  its  outputs,  how  the  project  outcomes  will  affect  the 
stakeholder,  and  the  stakeholder’s  expectations  about  why,  how,  and  when  FVAP  will 
undertake  the  project.  Moreover,  in  turning  to  stakeholders  as  experts,  with  an  explicit 
role  in  the  expression  of  needs,  the  development  of  the  approach,  and  the  specification 
of  indicators  (metrics  and  measures)  and  criteria  (thresholds  or  ranges),  FVAP  might 
be  able  to  leverage  their  capability  and  capacity.  If  FVAP  does  not  choose  to  engage 
stakeholders,  it  should  be  able  to  explain  why. 

The  worksheet  also  calls  for  a  notional  analysis  of  anticipated  benefits  and  costs 
to  FVAP,  its  stakeholders,  and  the  voting  assistance  system  writ  large.6  We  use  the 
term  notional  to  acknowledge  the  challenges  and  limitations  of  strict  reliance  on  quan- 


4  In  effect,  it  should  be  possible  to  construct  a  rough  logic  model  for  the  project  that  depicts  the  relationship 
between  the  proposed  project  and  an  intended  outcome. 

5  Stakeholders  with  an  interest  in  a  project  might  reside  within  FVAP  or  be  external  to  FVAP.  External  stake¬ 
holders  include  other  DoD  and  federal  agencies,  such  as  DOJ,  EAC,  and  NIST;  state  and  local  election  officials; 
NGOs;  voters;  technologists;  and  academics. 

6  Some  benefits  and  costs  might  be  intentional  (i.e.,  tied  to  intended  outcomes  and  actions),  and  others  might 
be  unintentional  (i.e.,  a  by-product  of  the  project). 
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tification.  FVAP  might  lack  access  to  the  data  or  tools  to  develop  a  fully  quantitative 
assessment;  moreover,  some  anticipated  benefits  (e.g.,  goodwill)  might  be  less  amenable 
to  quantification  than  others,  such  as  increased  voting  success.  To  ignore  the  former 
might  be  to  produce  a  partial — and  possibly  misleading — analysis  of  the  project.  In 
those  instances,  FVAP  staff  can  set  out  the  benefits  or  costs  qualitatively,  with  some 
explanation.7 

Similarly,  it  might  be  difficult  to  quantify  risk.  Table  D.l  presents  a  DoD- 
originating  method  for  gauging  risk  (Greenfield  and  Camm,  2005,  citing  U.S.  mili¬ 
tary  doctrine)  that  frames  risk  in  terms  of  the  severity  (e.g.,  negligible  to  catastrophic) 
and  probability  (e.g.,  unlikely  to  frequent)  of  a  hazard  that  entails  negative  or  bad  con¬ 
sequences.  In  this  setting,  the  bad  consequences  might  include  project  failure  (itself, 
potentially  costly)8  or  adverse  outcomes  for  FVAP  or  DoD  operations  and  for  stake¬ 
holders  and  the  voting  assistance  system  writ  large.  The  sources  of  risk  in  FVAP  proj¬ 
ects  could  include  untried  methodologies,  suggesting  the  importance  of  awareness  of 
similar  projects  and  their  results;  insufficient  resources  or  stakeholder  involvement; 
and  delays.  FVAP  staff  might  be  expected  to  consider  whether  it  is  possible  to  mitigate 
intolerable  risks  and  at  what  cost. 

Risk,  thus  conceived,  can  be  framed  broadly,  as  high,  medium,  or  low.  In  deciding 
whether  a  project  is  potentially  viable,  FVAP  staff  might  be  expected  to  estimate  or  set 
out  the  parameters  of  the  anticipated  net  payoff  (i.e.,  benefits  minus  costs)  with  some 
adjustment  for  risk.  If  the  balance  is  positive,  the  project  can  remain  in  play;  if  not,  it 
cannot.  Finally,  recognizing  that  projects  entail  opportunity  costs,  the  risk- adjusted  net 
payoff  of  the  project  should  look  at  least  as  good  as  those  of  the  alternatives.9 

Table  D.l 

Assessing  Severity  and  Probability 


Probability 


Severity  Frequent  Likely  Occasionally  Seldom  Unlikely 


SOURCE:  Greenfield  and  Camm,  2005. 


7  We  provide  an  example  of  this  approach  in  Chapter  Six,  in  our  assessment  of  the  VAO  training  program. 

8  Failures — and  midcourse  corrections — can  change  project  costs,  e.g.,  by  introducing  termination  or  redirec¬ 
tion  costs. 

9  We  return  to  the  issue  of  comparative  net  payoffs  in  the  discussion  of  the  decision  to  engage  and  in  Figure  D.4. 
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Lastly,  the  needs  that  motivate  a  project  can  potentially  be  met  by  another  agency 
or  by  a  private-sector  entity  (e.g.,  an  NGO)  in  some  other  way.  Another  organiza¬ 
tion  might  offer  a  better  way  to  meet  the  need,  or  the  project  might  be  better  aligned 
with  another  agency’s  mission.  Under  these  circumstances,  FVAP  staff  should  consider 
whether  encouraging  and  supporting  the  other  agency  to  meet  the  need  is  the  better 
decision. 

If  better  alternatives  appear  to  exist,  FVAP  should  also  consider  the  implications 
of  not  engaging  in  the  project.  It  should  be  possible  to  describe  and  consider  the  dif¬ 
ferences  that  deferring  to  another  agency  or  entity  rather  than  engaging  in  the  project 
would  make  to  FVAP’s  mission,  requirements,  or  operation  and  to  the  voting  system. 

In  supporting  the  director’s  decision  on  readiness,  the  worksheet  provides  guidance 
on  goals,  indicators,  and  criteria;  it  explores  sustainability,  and  it  addresses  planning. 

Absent  a  clear  statement  of  specific  and  observable  goals,  with  corresponding 
indicators  and  criteria,  FVAP  will  lack  the  ability  to  determine  whether  the  project  is 
on  or  off  course  and  potentially  requires  a  midcourse  adjustment  or,  in  the  extreme, 
termination.  Indicators  and  criteria  can  be  all  or  nothing,  by  degree,  quantitative  or 
qualitative,  and  time-dependent.  Because  individual  goals  can  be  tied  to  potential  haz¬ 
ards  and  risks,  indicators  can  serve  to  identify  the  emergence  of  a  risk.  Indicators  can 
also  be  formulated  to  monitor  relationships  with  stakeholders  and  to  reflect  stakehold¬ 
ers’  interests.10 

Resources  are  needed  to  see  a  project  through  to  completion  and  to  the  point  of 
meeting  needs  (e.g.,  transferring  project  results  to  customers  and  partners).  Resources 
can  include  funding,  staff,  data,* 11  infrastructure,  management  focus,  DoD  policy  sup¬ 
port,  stakeholder  support,  and  stakeholder  goodwill.  By  identifying  all  needed  resources 
in  advance,  the  analysis  can  build  confidence  that  the  project  is  sustainable.  Fiowever, 
identifying  needed  resources  does  not  guarantee  their  availability.  If  a  resource’s  avail¬ 
ability  cannot  be  fully  determined,  the  availability  of  the  needed  resource  constitutes 
a  risk.  For  example,  if  DoD  policy  support  is  required  to  succeed  with  a  project  and 
developing  that  policy  support  is  part  of  the  project,  then  DoD  policy  support  should 
be  identified  as  a  potential  source  of  risk.  Obtaining  DoD  policy  support  might  also  be 
an  interim  goal  of  the  project  and,  if  not  obtained,  used  to  decide  to  abandon  further 
work  on  the  project.  Similarly,  if  the  project  requires  stakeholder  support  but  that  sup¬ 
port  has  not  yet  been  secured,  it  should  be  identified  as  a  risk.12 


10  As  above,  FVAP  can  assess  similar  work  for  insight  to  the  construction  of  indicators  and  criteria. 

11  Data  might  be  necessary  to  gauge  a  project’s  progress.  For  example,  data  might  be  needed  to  define  the 
baseline  for  an  indicator.  By  considering  whether  such  data  are  available  or  can  be  collected,  the  analysis  might 
conclude  that  the  project  cannot  be  sustained  (e.g.,  it  might  not  be  possible  to  establish  a  baseline  for  a  success 
indicator  of  the  project).  Uncertainties  about  the  sources  or  methods  for  collecting  data  also  represent  risks.  As 
with  other  resources,  the  risks  associated  with  data  requirements  might  also  suggest  reexploration  of  the  risk 
assessment. 

12  If  the  analysis  surfaces  significant  new  risks,  then  the  risk  analysis  (see  above)  should  be  revisited. 
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Finally,  FVAP  must  have  a  plan  for  managing  the  project  through  and  beyond 
completion.  Plans  identify  who  will  be  responsible  for  the  project,  including  determi¬ 
nations  about  the  transfer  (distribution  and  use)  of  results,  midcourse  corrections,  and 
project  termination.  They  also  identify  how  and  when  such  determinations  will  be 
made.  The  project’s  goals,  indicators,  and  criteria  can  be  used  to  help  make  these  deter¬ 
minations.  Plans  can  also  consider  establish  procedures  for  appropriate  transparency, 
including  the  content  and  timing  of  reporting  and  dissemination,  and  for  integrating 
outputs  into  FVAP  operations. 

Projects  might  produce  results  that  need  to  be  integrated  into  FVAP  operations 
and  other  activities  upon  completion.  For  example,  answering  a  question  might  suggest 
refinements  to  an  ongoing  activity.  Building  a  new  capability  or  increasing  an  existing 
capacity  for  FVAP  will,  almost  by  definition,  require  a  plan  for  its  incorporation  into 
FVAP’s  operations.  A  complete  plan  would  describe  what  will  be  integrated  or  trans¬ 
ferred,  to  whom,  and  under  what  circumstances  upon  completion  of  the  project.  Note 
that  such  a  transfer  might  need  to  make  assumptions  about  the  resources  available  in 
the  receiving  entity,  and  such  an  assumption  might  surface  as  resource  constraint  that 
requires  reconsideration. 

Boxes  D.l  through  D.8  present  the  elements  of  the  worksheets  that  correspond  to 
questions  1  through  8  in  the  director’s  checklist. 


The  Decision  to  Engage 

The  checklist  and  worksheet  provide  tools  in  the  form  of  a  series  of  questions  and  sup¬ 
porting  analyses  to  determine  whether  FVAP  should  engage  in  a  project  and  whether 
it  is  ready  to  do  so.  If  the  director  can,  on  the  basis  of  the  analyses,  answer  “yes”  to  the 
first  eight  questions,  he  or  she  should  have  confidence  that  the  project  holds  promise; 
otherwise,  not.  Thus,  these  tools  can  also  be  used  to  filter  projects. 

A  project  might  pass  through  the  initial  filter,  but  the  director  must  still  deter¬ 
mine  whether,  in  the  larger  context  of  other  agency  activities  and  resource  constraints, 
the  agency  should  engage  in  the  project.  We  approach  this  issue  with  two  additional 
questions: 

•  Flow  does  this  project  stack  up  against  others? 

•  Is  it  a  project  that  FVAP  can  do  well? 

The  first  question  takes  into  account  FVAP’s  priorities  and  the  relative  merits  of 
the  proposed  and  other  projects.  FVAP’s  director  sets  its  priorities.  Judging  the  merit 
of  a  viable  project  involves  considering  whether  FVAP  could  be  doing  something 
better  with  its — and  others’ — resources.  When  resources  are  limited,  alignment  with 
the  agency’s  priorities  becomes  especially  important.  This  consideration  also  applies 
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Box  D.1 

Will  the  Project  Meet  a  Need? 


1.  Characterize  the  need  that  the  project  will  meet. 

a.  State  the  need  succinctly  and  nontechnically. 

b.  Specify  the  questions  the  project  will  answer,  capabilities  it  will  create,  or  capacities  it  will 
increase. 

i.  Discuss  the  difference  between  the  current  and  intended  state  of  knowledge, 
capability,  or  capacity. 

ii.  Discuss  whether  and  how  answering  questions,  creating  capabilities,  or  increasing 
capacities  will  depend  on  stakeholder  actions. 

c.  Explain  how  answering  these  questions,  creating  these  capabilities,  or  increasing  these 
capacities  will  lead  to  attaining  an  outcome  in  support  of  FVAP's  mission,  meeting  its 
requirements,  or  better  managing  its  operations. 

i.  Describe  the  plausible  path  (involving  outputs,  transfer,  and  stakeholders'  use) 
by  which  the  project  will  attain  an  outcome,  meet  a  requirement,  or  assist  with 
managing  operations. 

ii.  Discuss  whether  and  how  attaining  outcomes,  meeting  a  requirement,  or  assisting 
with  managing  operations  will  depend  on  stakeholder  actions. 

2.  Provide  evidence  of  the  need. 

3.  Characterize  the  role  that  appropriate  experts  have  played  in  conceptualizing  the  project. 

a.  Identify  the  experts  who  have  been  consulted  for  advice  or  other  feedback  both  within 
and  external  to  FVAP. 

b.  Describe  their  credentials  and  interests  in  the  project. 

c.  Summarize  their  advice  and  feedback. 

d.  Explain  how  their  input  has  been  used,  thus  far,  to 

i.  state  the  need 

ii.  refine  questions,  capabilities,  or  capacities 

iii.  refine  the  plausible  path  to  attaining  an  outcome,  meeting  a  requirement,  or  helping 
manage  operations 

iv.  identify  dependencies  on  stakeholders. 

e.  If  their  input  has  not  been  elicited  or  used,  explain  why  not. 


to  stakeholders.  A  project  might  depend  on  the  commitments  and  resources  of  other 
organizations.  Thus,  the  project  needs  to  be  well  aligned  with  the  priorities  of  such 
stakeholders.  Moreover,  a  viable  project  that  aligns  with  priorities  might  be  one  of 
several  well-aligned,  viable  projects  under  consideration.  Such  a  project  will  also  take 
place  in  the  context  of  ongoing  operations  of  FVAP  and  its  stakeholders.  Thus,  FVAP 
should  consider  how  a  project  stacks  up  against  other  candidate  projects. 

A  simple  two-by-two  matrix  can  be  used  to  conceptualize  the  use  of  project  risk 
and  net  payoff  to  compare  projects  (Figure  D.4).  Risk  and  net  payoff  can  be  gauged  as 
high  or  low.  A  high-net  payoff  project  is  one  whose  benefits  considerably  outweigh  its 
costs.  A  low-net  payoff  project  is  one  whose  benefits  are  more  closely  matched  to  its 
costs.  The  risks  associated  with  either  type  of  project  can  be  high  or  low. 

Given  our  understanding  of  FVAP,  its  mission,  and  its  risk  tolerance,  we  suggest 
that  it  first  consider  low-risk,  high-net  payoff  projects  (upper-right  quadrant)  and  rule 
out  high-risk,  low-net  payoff  projects  (lower-left  quadrant).  Choices  among  multiple 
low-risk  and  high-net  payoff  projects  (such  as  A,  B,  and  C)  will  require  a  more  careful 
examination  of  risks  and  net  payoffs  to  identify  salient  differences  that  can  be  used  to 
rank  the  projects. 
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Box  D.2 

Is  the  Approach  Fit  for  Its  Purpose? 


1.  Characterize  the  approach  the  project  will  take  to  answer  questions,  create  capabilities,  or 
increase  capacities  and  bring  results  to  bear  on  outcomes,  requirements,  and  operations. 

a.  Describe  succinctly  and  nontechnically  the  approach  that  the  project  will  take. 

i.  Identify  the  project  outputs,  such  as  data,  analysis  tools,  training  materials,  or 
reports. 

ii.  Explain  how  the  project  will  produce  those  outputs. 

iii.  Explain  how  the  project  or  others  will  use  the  outputs  to  fill  the  need  and  attain 
outcomes,  meet  requirements,  or  help  manage  operations. 

b.  Describe  any  additional  steps  (e.g.,  involving  transfer  or  use)  that  must  be  taken  to 
ensure  that  answers,  capabilities,  or  capacities  and  outputs  lead  to  desired  outcomes, 
meet  requirements,  or  help  manage  operations. 

c.  Determine  (e.g.,  through  consideration  of  law  and  policy)  whether  FVAP  has  the 
authority  to  see  the  project  through  to  completion  and  to  the  point  of  meeting  needs. 

d.  If  FVAP  lacks  authority,  determine  whether  it  can  create  the  incentives  necessary  for 
others  to  do  so. 

2.  Identify  similar  work  that  is  under  way  or  has  been  undertaken  previously,  either  at  FVAP  or 
elsewhere,  and  discuss  the  implications  for  this  project. 

a.  Describe  succinctly  and  in  nontechnical  terms  the  approach  that  the  work  took  or  is 
taking. 

b.  Explain  why  the  work  is  relevant,  considering  the 

i.  needs  addressed 

ii.  approach  taken 

iii.  outputs  and  outcomes 

iv.  causes  of  success  or  failure. 

c.  Discuss  any  lessons  that  can  be  drawn  from  that  work  for  the  completion  and  success  of 
this  project. 

d.  If  similar  prior  projects  did  not  succeed,  explain  why  not  and  what  will  be  different  about 
this  project. 

e.  Explain  how  lessons  have  informed  this  project's  approach. 

3.  Characterize  the  role  that  appropriate  experts  have  played  in  developing  the  approach. 

a.  Identify  the  experts  who  have  been  consulted  for  advice  or  other  feedback  both  within 
and  external  to  FVAP. 

b.  Describe  their  credentials  and  interests  in  the  project. 

c.  Summarize  their  advice  and  feedback. 

d.  Explain  how  their  input  has  been  used,  thus  far,  to 

i.  refine  the  approach 

ii.  identify  relevant  prior  work 

iii.  interpret  relevant  prior  work. 

e.  If  their  input  has  not  been  elicited  or  used,  explain  why  not. 


The  state  of  FVAP’s  core  capabilities  and  attendant  capacity  might  also  be  con¬ 
sidered  in  making  the  decision  to  engage  in  projects  that  are  well  aligned  with  FVAP 
and  stakeholder  priorities  and  have  the  potential  for  high  net  payoffs  at  low  risk.  The 
director  might  need  to  assess  the  maturity  of  such  capabilities,  their  sufficiency  (i.e.,  the 
amount  of  headway  in  the  agency  that  would  be  needed  to  absorb  the  new  project),  and 
the  agency’s  track  record  of  execution  in  this  arena  or  project  area. 


Sources  of  Ideas 

We  drew  on  prior  work  in  other  venues  to  formulate  the  tools.  Here  we  describe  the 
most-relevant  lessons  from  that  work  and  how  we  applied  them. 
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Box  D.3 

Will  the  Right  Stakeholders  Care  About  the  Project  for  the  Right  Reasons? 


1.  Characterize  the  stakeholders  and  their  interests,  and  discuss  the  implications  for  this 
project. 

a.  Identify  the  stakeholders  with  an  interest  in  this  project  considering  those 

i.  within  FVAP 

ii.  external  to  FVAP  (e.g.,  other  DoD  and  federal  agencies,  election  officials,  NGOs, 
academics,  voters). 

b.  Describe  their  interests. 

2.  Describe  how  stakeholders  will  be  expected  to  respond  to  the  project,  considering 

a.  how  they  might  use  or  share  information  about  the  project  and  its  outputs 

b.  how  the  outcomes  will  affect  them 

c.  their  expectations  about  why,  how,  and  when  FVAP  will  undertake  the  project. 

3.  Characterize  which  stakeholders  are  important  and  why,  considering 

a.  why  their  interests  matter 

b.  whether  the  project  will  advance  or  obstruct  their  interests 

c.  whether  FVAP  will  need  to  manage  their  expectations. 


Selection,  Management,  and  Completion  of  Projects 

Given  the  initial  emphasis  on  research-related  projects,  we  began  our  review  with  meth¬ 
ods  and  tools  used  to  select  and  evaluate  research  projects.  Perhaps  the  best-known  and 
most  popularized  framework  is  George  Heilmeier’s  “Catechism”  (Shapiro,  1994).  Heil- 
meier  developed  nine  questions  that  he  and  others  successfully  used  to  screen  proposed 
research  projects.  Heilmeier  was  a  former  director  of  the  Defense  Advanced  Research 
Projects  Agency  and  a  researcher  (he  was  the  cocreator  of  liquid-crystal  display  tech¬ 
nology).  His  nine  questions  are  sequential  and  require  little  interpretive  context: 

1.  What  are  you  trying  to  do?  Articulate  your  objectives  using  absolutely  no  jargon. 

2.  How  is  it  done  today,  and  what  are  the  limits  of  the  current  practice? 

3.  What  is  new  in  your  approach,  and  why  do  you  think  it  will  succeed? 

4.  Who  cares? 

5.  If  you  are  successful,  what  difference  will  it  make? 

6.  What  are  the  risks  and  the  payoffs? 

7.  How  much  will  it  cost? 

8.  How  long  will  it  take? 

9.  What  are  the  midterm  and  final  exams  to  check  for  success? 


Heilmeier’s  questions  provided  inspiration  for  the  first  eight  questions  in  our 
checklist.  Heilmeier’s  emphasis  on  simple  articulation,  prior  work,  stakeholders,  risks 
and  payoffs,  plans,  and  monitoring  execution  are  all  incorporated  into  the  checklist 
questions.  Our  approach  reorders  and  combines  some  of  Heilmeier’s  questions.  In  sev¬ 
eral  of  our  questions,  we  included  his  emphasis  on  understanding  the  relationship  to 
prior  work  (what  is  new).  Heilmeier’s  questions  are  intended  to  screen  the  projects  to 
identify  promising  options.  Like  our  checklist  and  worksheet,  they  also  require  look¬ 
ing  downstream  to  understand  how  a  project  should  be  monitored  and  success  judged 
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Box  D.4 

Do  the  Benefits  of  the  Project  Justify  the  Costs  and  Risks  to  the  Federal  Voting  Assistance 
Program  and  the  Voting  Assistance  System? 


1 .  Assess  the  anticipated  benefits  of  the  project. 

a.  List,  describe,  and,  to  the  extent  possible,  account  for  the  benefits  of  the  project, 
including  attaining  outcomes,  meeting  requirements,  improving  operations,  satisfying 
stakeholder  objectives,  improving  relationships  with  stakeholders,  and  garnering 
goodwill.  Not  all  benefits  are  quantifiable. 

b.  Identify  the  beneficiaries,  both  within  FVAP  and  external  to  FVAP  (i.e.,  in  the  larger 
voting  assistance  system). 

c.  Identify  which  benefits  are  intentional  (meeting  stated  needs)  and  unintentional  (by¬ 
products  of  the  project). 

2.  Assess  the  anticipated  costs  of  the  project. 

a.  List,  describe,  and,  to  the  extent  possible,  account  for  the  resource  needs  of  the  project, 
including  funding,  staff,  infrastructure,  management  focus,  DoD  policy  support,  and 
stakeholder  support  and  goodwill.  Not  all  costs  are  quantifiable. 

b.  Identify  which  costs  are  borne  by  FVAP  or  by  other  participants  in  the  voting  assistance 
system. 

c.  Identify  which  costs  are  intentional  and  unintentional. 

d.  Explain  how  failure  of  the  project  would  change  the  costs. 

e.  Describe  any  costs  that  must  be  borne  to  make  midcourse  corrections  or  take  an  off-ramp 
and  terminate  the  project. 

3.  Determine  whether  the  anticipated  net  payoff  (i.e.,  benefits  minus  costs)  is  positive,  either 
quantitatively  or  conceptually. 

4.  Determine  whether  this  is  a  high-  or  low-risk  project. 

a.  Characterize  risk  in  terms  of  potential  hazards  or  bad  consequences,  probability,  and 
severity. 

i.  Identify  potential  bad  consequences,  including  the  possibilities  of  failure  and  damage 
to  FVAP,  leadership,  and  the  voting  assistance  system. 

ii.  Assess  the  probability  of  realizing  those  consequences,  ranging  from  unlikely  to 
frequent. 

iii.  Assess  the  severity  of  those  consequences,  ranging  from  negligible  to  catastrophic. 

b.  Identify  key  sources  of  risk  (e.g.,  untried  methodology,  insufficient  resources  or 
stakeholder  involvement,  and  delays). 

c.  Describe  whether  it  is  possible  to  mitigate  intolerable  risk  (e.g.,  risks  that  would  cause  the 
project  to  fail,  severely  affect  FVAP  or  other  DoD  operations,  or  severely  affect  the  voting 
assistance  system)  and  at  what  cost. 

5.  Determine  whether  the  risk-adjusted  net  payoff  is  positive,  considering 

a.  costs  of  failure,  including  loss  of  goodwill 

b.  costs  of  risk  mitigation 

c.  benefits  in  light  of  the  likelihood  of  achieving  them. 

6.  Compare  the  net  payoffs  with  those  of  alternative  uses  of  resources. 


Box  D.5 

Is  the  Federal  Voting  Assistance  Program  the  Best  Agency  to  Engage  in  the  Project  in  This 
Way? 


1.  Identify  other  agencies  or  entities  that  could  meet  the  needs,  considering  whether 

a.  the  project  would  be  better  aligned  with  the  mission  of  one  of  these  agencies  or  entities 

b.  such  an  agency  or  entity  could  meet  the  need  at  lower  cost  or  risk  and  how  engaging 
another  agency  or  entity  might  alter  the  amount  or  distribution  of  benefits 

c.  that  agency  or  entity  would  be  willing  or  could  be  induced  to  undertake  the  project. 

2.  Explain,  in  view  of  the  alternatives,  why  FVAP  is  the  best  agency  to  meet  the  need. 

3.  Describe  the  difference  it  will  make  to  FVAP's  mission,  requirements,  or  operation  and  the 
voting  assistance  system  if  FVAP  does  not  engage  in  this  project. 
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Box  D.6 

Will  the  Federal  Voting  Assistance  Program  Know  Whether  It  Is  Succeeding? 


1.  Characterize  the  goals  of  the  project. 

a.  List  and  describe  the  goals,  and  evaluate  whether  they  are  specific  and  observable. 

b.  Explain  how  meeting  the  goals  will  contribute  to 

i.  answering  the  question,  creating  the  capability,  or  increasing  the  capacity 

ii.  attaining  the  intended  outcome,  meeting  the  intended  requirement,  or  helping 
manage  operations  of  the  project. 

2.  Identify  the  indicators  (metrics  and  measures)  and  criteria  (thresholds  or  ranges)  that  will  be 
used  to  monitor  and  assess  progress,  considering  whether  an  indicator  is  all  or  nothing  or  by 
degree,  quantitative  or  qualitative,  or  time-dependent. 

3.  Explain  how  the  indicators  and  criteria  will  address  the  stated  goals,  considering  whether 
they  can  be  used  to 

a.  trace  progress  toward  answering  questions,  creating  capabilities,  increasing  capacities, 
attaining  outcomes,  or  otherwise  furthering  the  FVAP  mission 

b.  monitor  risks 

c.  decide  whether  and  when  to  make  midcourse  corrections  or  take  an  off-ramp  and 
terminate  the  project 

d.  assess  outputs  and  outcomes 

e.  monitor  relationships  with  affected  stakeholders 

f.  reflect  stakeholders'  interests. 

4.  Identify  and  compare  uses  of  the  indicators  and  criteria  in  similar  circumstances  by  FVAP  or 
others. 

a.  Describe  the  purpose  of  the  indicators  in  the  other  environment. 

b.  Discuss  any  lessons  that  can  be  drawn  from  other  current  or  prior  uses  of  the  indicators 
and  criteria. 

c.  If  current  or  prior  uses  have  not  been  fruitful,  explain  why  not  and  what  will  be  different 
about  this  project. 

5.  Characterize  the  role  that  appropriate  experts  have  played  in  developing  the  indicators  and 
criteria  for  this  project. 

a.  Identify  the  experts  who  have  been  consulted  for  advice  or  other  feedback  both  within 
FVAP  and  external  to  FVAP. 

b.  Describe  their  credentials  and  interests  in  the  project. 

c.  Summarize  their  advice  and  feedback. 

d.  Explain  how  their  input  has  been  used  thus  far  to 

i.  formulate  indicators  and  criteria  for  this  project 

ii.  identify  relevant  prior  work 

iii.  interpret  relevant  prior  work. 

e.  If  their  input  has  not  been  elicited  or  used,  explain  why  not. 


Box  D.7 

Is  the  Project  Sustainable? 


1.  Determine  whether  FVAP  has  access  to  resources  needed  to  see  the  project  through  to 
completion  and  to  the  point  of  meeting  needs,  including  funding,  staff,  infrastructure, 
management  focus,  DoD  policy  support,  and  stakeholder  support  and  goodwill. 

a.  If  DoD  policy  support  is  required  but  not  yet  secured,  identify  policy  support  as  a  risk. 

b.  If  stakeholder  support  is  required  but  not  yet  secured,  identify  stakeholder  support  as  a 
risk. 

c.  If  other  resources  have  not  yet  been  secured,  identify  the  needs  for  those  resources  as 
risks. 

2.  Determine  whether  FVAP  has  the  data  it  will  need  to  gauge  progress,  considering 

a.  data  to  define  the  baseline  of  an  indicator 

b.  the  sources  and  methods  of  data  collection  that  are  needed  to  observe  the  indicators. 
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Box  D.8 

Does  the  Federal  Voting  Assistance  Program  Have  a  Plan  for  Managing  the  Project? 


1.  Identify  who  (in  FVAP)  is  or  will  be  responsible  for  project  formulation,  implementation, 
oversight,  and  transfer,  including  making  determinations  about  the  distribution  and  use  of 
results,  midcourse  corrections,  or  project  termination. 

2.  Identify  the  points  in  the  life  of  the  project  when  FVAP  will  apply  the  indicators  and  criteria 
to  decide  whether  to  make  midcourse  corrections  or  take  an  off-ramp  and  terminate  the 
project. 

3.  Describe  how  FVAP  will  ensure  appropriate  transparency  over  the  life  of  the  project, 
beginning  with  formulation. 

4.  Describe  how  the  results  will  be  reported  on  and  disseminated  to  stakeholders  upon 
completion  of  the  project. 

5.  Describe  how  the  results  will  be  integrated  into  FVAP  operations  and  other  activities. 


Figure  D.4 

Risk  Versus  Net  Payoff  as  a  Basis  for  Project 
Comparisons 


RAND  RR882-D.4 


(i.e.,  Heilmeier’s  question  9).  His  questions  suppress  details  of  supporting  analyses  but 
imply  the  need  for  analysis. 

We  referred  to  the  U.S.  Department  of  Transportation’s  Office  of  Research  and 
Development  Evaluation  Implementation  Plan  (Federal  Railroad  Administration 
[FRA],  2013)  as  a  detailed  example  of  how  another  federal  agency  (FRA)  evaluates  the 
research  that  it  sponsors.  This  work  provided  us  with  a  concrete  example  of  a  research 
checklist  and  the  process  for  applying  it.  The  FRA  plan  distinguishes  two  uses  of 
research  and  development  evaluations:  guiding  and  planning  research  and  develop¬ 
ment  implementation  to  improve  a  program  of  research  (formative)  and  seeking  to 
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prove  or  demonstrate  a  research  program’s  success  (summative).  We  formulated  our 
approach  with  formative  research  projects  in  mind,  but  some  FVAP  projects  might  be 
undertaken  to  demonstrate  the  success  of  an  FVAP  program  (summative). 

These  two  applications  of  FRA’s  research  and  development  evaluation  imple¬ 
mentation  plan  identify  four  types  of  evaluation:  context,  input,  implementation,  and 
impact.  Context  evaluation  assesses  a  program’s  external  factors  (such  as  need,  assets, 
problem,  and  opportunities)  that  assist  in  formulating  goals  of  a  project.  This  view 
helped  to  shape  several  of  the  questions  used  in  our  checklist  (e.g.,  needs,  resources, 
and  goals).  Input  evaluation  assesses  alternative  approaches  that  serve  as  inputs  to  a 
program  (such  as  staffing  plans  and  budgets).  This  emphasizes  the  formulation  of  the 
research  approach  and  the  resources  it  needs  to  be  successfully  implemented.  Imple¬ 
mentation  evaluation  assesses  a  program’s  plan  and  execution  to  determine  how  well 
that  program  is  progressing.  This  is  related  to  our  question  dealing  with  indicators  and 
criteria.  Impact  evaluation  assesses  outcomes.  This  is  related  to  emphasis  on  outcomes 
in  the  first  several  questions  of  our  checklist.  Together,  the  cross-product  of  uses  and 
types  of  evaluation  creates  a  matrix  of  questions  FRA  uses  to  select  and  evaluate  proj¬ 
ects.  The  Department  of  Transportation’s  approach  is  more  structured  than,  but  con¬ 
ceptually  consistent  with,  Heilmeier’s  approach. 

We  also  referred  to  the  Measure  Evaluation  PRH  14-point  checklist  for  screening 
the  performance-management  plans  for  proposed  projects  (Measure  Evaluation  PRH, 
2012).  The  intent  of  this  checklist  and  its  supporting  analyses  is  to  surface  and  make 
decisions  about  characteristics  the  organization  believes  lead  to  successful  projects  or 
programs.  The  intent  is  similar  to  that  of  our  checklist  and  worksheet.  The  checklist 
enables  an  organization  to  think  about  and  plan  projects  involving  data  collection  pri¬ 
marily  related  to  health  care  and  the  impact  of  related  policies.  The  health  care  domain 
has  limited  overlap  with  FVAP’s  range  of  research  activities,  but  its  emphasis  on  proj¬ 
ects  with  data  collection  is  clearly  relevant  to  those  FVAP  projects  that  need  to  collect 
and  analyze  data  to  report  on  or  improve  FVAP  activities  and  its  impact  on  the  voting 
assistance  system. 

The  Measure  Evaluation  PRH  checklist  considers  costs,  indicators,  key  outputs 
and  their  relationships  to  desired  outcomes,  and  alignment,  as  do  Heilmeier  and  FRA. 
There  is  considerable  consistency  among  these  approaches,  which  gave  us  confidence 
that  we  were  not  overlooking  relevant  dimensions  in  formulating  our  checklist  and 
worksheet. 

Roles  and  Responsibilities 

Staff  and  management  play  different  roles  in  formulating,  executing,  and  evaluating 
research  projects.  Konig  et  ah,  2013,  provides  us  with  insights  about  how  contradictory 
values,  such  as  adaptability  and  stability,  can  be  supported  in  effective  research  pro¬ 
grams.  In  order  to  manage  such  tensions,  Konig  et  al.  extended  Quinn  and  Rohrbaugh’s 
Competing  Values  Framework  (Quinn  and  Rohrbaugh,  1983).  The  result  is  a  two- 
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dimensional,  four-quadrant  framework,  with  decentralization  and  centralization  jux¬ 
taposing  internal  and  external  focus.  By  locating  its  activities  in  one  of  these  quadrants, 
an  organization  can  focus  the  roles  needed  to  improve  its  effectiveness.  Each  quadrant 
identifies  several  appropriate  roles  and  responsibilities,  such  as  mentor,  facilitator,  inno¬ 
vator,  broker,  monitor,  coordinator,  producer,  and  director.  These  ideas  helped  us  to 
differentiate  the  roles  of  FVAP’s  director  and  staff  in  the  checklist  and  worksheet. 

Evaluation 

Most  of  our  references  emphasize  the  importance  of  early  consideration  of  project 
evaluation  (e.g.,  Heilmeier’s  exams).  We  consulted  Weiss,  1972,  on  survey  evaluation 
research  methods,  primarily  focused  on  experimental  research  programs,  to  identify 
ideas  that  might  be  applicable  to  FVAP,  given  its  work  on  surveys.  Weiss  discusses 
the  notions  of  formative  and  summative  research,  which  FRA’s  research  and  develop¬ 
ment  evaluation  implementation  plan  uses.  Weiss  also  emphasizes  that  program  goals 
need  to  be  articulated  and  indicators  need  to  be  defined  to  assess  whether  the  program 
achieves  its  goals. 

Weiss  discusses  three  standard  models  for  designing  evaluations  of  experimen¬ 
tal  research:  experimental,  quasi-experimental,  and  nonexperimental.  The  first  two 
approaches  assume  randomized  or  existing  groups  that  establish  control  and  treat¬ 
ment  groups.  The  last  approach  is  useful  when  control  and  treatment  groups  cannot 
be  defined.  Nonexperimental  evaluations  can  be  applied  before  and  after,  only  after,  or 
after  only  with  a  comparison  group.  Weiss  also  describes  the  ways  in  which  evaluations 
can  fail.  A  program  can  fail  to  establish  confidence  in  its  demonstration  of  a  causal 
relationship  between  the  program  variables  and  a  given  outcome  (internal  validity).  A 
program  can  fail  to  establish  confidence  that  the  program’s  external  variables  did  not 
affect  the  program’s  outcomes  (external  validity).  External  validity  enables  an  experi¬ 
ment’s  results  to  be  generalized  to  other  programs. 

These  ideas  support  the  emphasis  in  our  checklist  of  the  need  to  articulate  indi¬ 
cators  and  criteria  and  to  use  them  in  midterm  assessments  and  decisions  to  take  off¬ 
ramps.  Weiss  describes  the  standard  framework  that  the  scientific  community  uses  to 
evaluate  experimental  research.  The  different  facets  of  validity  might  provide  a  helpful 
framework  for  understanding  how  to  interpret  and  communicate  data  that  FVAP  col¬ 
lects  and  analyzes. 

Newcomer,  Hatry,  and  Wholey,  2010,  discusses  how  a  single  manager  or  manage¬ 
ment  team  responsible  for  a  program  can  use  program-evaluation  tools  to  systemati¬ 
cally  address  questions  about  program  operations  and  results.13  The  report  also  refers 
to  formative  and  summative  evaluations  and  to  factors  that  should  be  considered  in 


13  Programs  can  be  thought  of  as  a  set  of  resources  and  activities  directed  toward  one  or  more  common  goals. 
This  definition  emphasizes  the  alignment  and  management  of  a  set  of  projects  in  pursuit  of  a  mission,  such  as 
providing  voter  assistance. 
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choosing  indicators.  In  those  authors’  view,  program  evaluation  consists  of  several  key 
steps: 


•  scoping 

•  formulating  the  evaluation  objectives 

•  framing  evaluation  questions 

•  matching  methodologies  to  questions 

•  identifying  constraints  on  implementing  methodologies 

•  identifying  means  to  ensure  quality  of  work 

•  anticipating  problems  and  developing  contingency  plans. 

The  steps  are  aimed  at  understanding  whether  a  proposed  approach  meets  a  stated 
need  and  the  plan  for  demonstrating  whether  that  need  is  met.  These  ideas  help  us  to 
refine  our  notion  of  fitness  for  purpose.  The  need  for  the  project  must  be  clear  and 
relevant,  and  the  approach  must  be  appropriate  to  meeting  that  need.  The  steps  also 
highlight  the  importance  of  a  management  plan  that  anticipates  potential  problems 
and  provides  off-ramps  that  can  be  taken  based  on  interim  evaluation  results.  This 
work  provided  yet  one  more  check  on  the  relevance  of  the  questions  we  included  in  our 
proposed  tools. 

Evaluation  methods  have  pitfalls.  Hatry  and  Newcomer,  2010,  discusses  the  com¬ 
ponents  of  methodological  integrity  (measurement  validity  and  internal  and  external 
validity).  It  makes  the  point  that  the  failure  to  establish  credibility  (the  findings  are 
believable  to  the  intended  audience)  can  undermine  other  aspects  of  methodological 
integrity.  It  describes  specific  pitfalls  that  a  researcher  might  encounter.  The  authors 
group  these  pitfalls  according  to  when  they  occur:  before  data  collection,  during  data 
collection,  or  after  data  collection.  These  ideas  are  closely  related  to  the  experimental 
research  standards  that  Weiss,  1972,  discusses  and  might  also  be  useful  to  FVAP. 

The  pitfalls  before  data  collection  that  Hatry  and  Newcomer,  2010,  identifies 
include  the  following: 

•  failure  to  access  whether  the  program  is  evaluable 

•  use  of  inadequate  indicators  of  program  outcomes 

•  failure  to  secure  input  from  stakeholders  on  appropriate  evaluation  criteria 

•  failure  to  clarify  program  managers’  expectations  on  what  can  be  learned  from 

the  evaluation 

•  failure  to  pretest  data-collection  instruments  appropriately 

•  inadequately  training  data  collectors. 

We  used  these  ideas  to  formulate  our  checklist  and  worksheet.  We  can  draw 
several  lessons  from  these  pitfalls.  The  first  and  second  emphasize  the  need  to  think 
through,  before  engaging  in  a  project,  how  the  team  will  evaluate  the  project — one  of 
the  reasons  we  emphasize  indicators  and  criteria  in  our  checklist  and  worksheet.  Hatry 
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and  Newcomer,  2010,  also  notes  the  importance  of  consulting  with  SMEs  and  stake¬ 
holders  when  formulating  indicators  and  criteria  to  be  used  in  evaluation.  The  last  two 
pitfalls  are  most  relevant  to  FVAP  projects  that  collect  and  analyze  data.  Pretesting  and 
training  pertain  to  any  project  that  involves  surveying. 

Several  of  our  references  emphasize  the  need  for  stakeholder  acceptance  of  the 
need  for  a  project,  the  means  of  execution,  and  interpretation.  Transparency  can  play 
a  part  in  establishing  a  project’s  credibility  with  stakeholders.  Hahnel,  2013,  describes 
methods  for  publicizing  intermediate  research  results.  Although  Hahnel,  2013,  is 
focused  primarily  on  data-management  plans  for  data  collected  by  research  projects, 
the  intent  of  doing  so  is  relevant  to  less  empirical  projects.  This  supports  our  view 
that  systematic  attention  to  transparency,  across  the  spectrum  of  stakeholder  positions, 
might  help  to  manage  criticism  and  build  credibility. 

Soliciting  and  Funding  Research 

FVAP  and  others  use  Broad  Agency  Announcements  (BAAs)  to  solicit  and  fund 
research  proposals.  We  examined  the  guidance  for  several  agency  processes.  The  U.S. 
Department  of  Commerce’s  procedural  description  of  the  use  of  BAAs  provided  insight 
to  contracting  details  (U.S.  Department  of  Commerce,  2014).  Of  particular  note  is 
its  discussion  of  required  BAA  content  and  evaluation  requirements.  Although  the 
Department  of  Commerce  does  not  describe  how  to  do  evaluations,  its  best  practices 
give  prominence  to  the  need  to  think  about  and  define,  before  a  project  is  selected,  how 
the  project  will  be  evaluated — again,  reinforcing  the  need  for  indicators  and  criteria. 
It  also  stresses  the  use  of  peer  reviews  of  submitted  proposals.  Such  reviews  are  proce¬ 
dural  but  also  highlight  the  role  outside  experts  can  play  in  formulating  and  evaluat¬ 
ing  research.  DHRA  also  has  a  template  for  creating  BAAs  (USD[P&R],  2011).  It  is 
largely  silent  on  how  to  identify  and  evaluate  high-quality  research  proposals,  but  it 
does  outline  a  review  process  and  the  role  that  nongovernment  SMEs  play  in  that  pro¬ 
cess.  These  references  are  relevant  to  any  BAA  that  FVAP  might  issue  in  the  future. 


Concluding  Remarks 

In  this  appendix,  we  proposed  a  checklist  for  the  director  and  a  worksheet  for  staff 
analysis  as  tools  to  help  FVAP  make  decisions  about  project  viability,  readiness,  and 
ultimate  engagement — including  project  modifications,  continuance,  and  curtailment. 

We  envision  the  decisionmaking  process  as  entailing  a  dialogue  between  FVAP’s 
director  and  staff.  Sequencing  might  enable  FVAP  to  stage  and  stop  efforts  as  the  exer¬ 
cise  proceeds.  The  effort  can  end  with  the  first  “no.”  Nevertheless,  the  analysis  will 
consume  resources,  possibly  both  FVAP’s  and  its  stakeholders’,  depending  on  their 
participation.  In  addition,  the  time  and  effort  could  be  spent  on  something  else.  Thus, 
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our  aim  was  a  process  that  could  be  dialed  up  or  down,  commensurate  with  the  size  of 
the  project  under  consideration. 

The  tools  also  highlight  the  role  that  stakeholders  and  SMEs  can  play  in  helping 
FVAP  make  decisions  about  starting,  changing,  or  stopping  projects.  In  some  instances, 
their  time,  attention,  and  other  resources  will  be  offered  freely  (within  reason)  to  FVAP 
because  of  their  interest  as  stakeholders  or  in  the  name  of  collegiality,  but,  in  other 
instances,  their  engagement  might  require  a  contract,  thus  implying  additional  cost. 


APPENDIX  E 


Training  Options  and  Recommendations 


FVAP  provides  training  in  person  and  online  to  VAOs,  IVA  office  staff,  and  others  in 
a  manner  that  is  consistent  with  law  and  as  set  out  in  policy,  but  it  has  some  latitude  to 
set  its  own  course  in  the  future  in  terms  of  the  content  of  training,  the  modes  of  train¬ 
ing,  and  the  nature  of  its  involvement  (Chapter  Four). 

The  law  does  not  specify  a  role  for  FVAP  in  VAO  or  IVA  office  training,  but  that 
lack  of  specificity  need  not  imply  a  lack  of  necessity  or  efficacy.  FVAP  serves  as  a  reposi¬ 
tory  of  knowledge  of  voting  assistance  and  tools  and  on  that  basis  alone  might  be  well 
positioned,  in  absolute  and  relative  terms,  to  play  a  significant  role  in  training.  The 
potential  for  ancillary  benefits,  addressed  in  Chapter  Six,  suggests  another  reason  for 
FVAP’s  involvement.  However,  whether  that  involvement  means  developing  training 
materials,  providing  in-person  training,  offering  online  training,  or  undertaking  some 
of  each  remains  to  be  determined. 

This  appendix  discusses  our  analysis  of  and  findings  on  FVAP’s  options  for  the 
VAO  training  program,  including  the  trade-offs  and  complementarities  among  train¬ 
ing  modes  (in  person,  online,  and  hybrid).  In  our  analysis,  we  apply  the  project-man¬ 
agement  tools,  presented  in  Appendix  D,  and  draw  heavily  from  our  site  visits  and 
training  observations,  from  the  literature  on  adult  learning  and  training  evaluation, 
and  from  additional  conversations  with  FVAP  staff  during  a  workshop  that  we  led. 


Framing  the  Options 

To  start,  we  turned  to  law  and  policy  to  set  out  the  range  of  options.1  In  summary,  we 
found  that  FVAP  faces  a  more  or  less  open  playing  field.  The  law  affords  considerable 
flexibility;  in  particular,  it  does  not  designate  a  provider  or  training  modes  and  meth¬ 
ods.  The  2012  DoD  instruction  assigns  FVAP  responsibility  for  conducting  training, 
appears  to  prefer  in-person  training,  and  establishes  the  frequency  and  timing  of  train- 


1  Although  we  frame  this  discussion  in  terms  of  options ,  we  note  that  FVAP  cannot  choose  a  course  in  isolation 
without  reference  to  or  consultation  with  other  parties  in  the  system. 
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ing  but  allows  a  combination  of  approaches,  depending,  for  example,  on  the  circum¬ 
stances  of  the  trainee. 

From  Chapter  Four: 

DoDI  1000.04,  2012,  calls  on  FVAP  to  “develop  and  deliver  multiple  types  of 
training  materials  for  use  by  IYA  offices,  IVAOs,  UVAOs,  and  recruiters”  and 
conduct  voting  assistance  training  during  even-numbered  years  worldwide,  and 
it  requires  that  IVAOs  and  UVAOs  complete  FVAP  training  prior  to  assuming 
duties.  Moreover,  the  instruction  indirectly  calls  for  the  availability  of  both  in- 
person  and  online  training  and  prefers  in-person  training.  It  requires  that  all  major 
command  VAOs,  IVAOs,  and  UVAOs  attend  FVAP  voting  assistance  workshops; 
asks  unit  commanders  to  provide  funding  to  enable  UVAOs  to  attend  in-person 
training  to  the  extent  practicable;  and  references  online  training  as  an  alternative 
for  VAOs  in  remote  locations. 

Given  the  lack  of  specificity  in  the  law  and  the  potential  to  revise  the  DoD  instruc¬ 
tion  (Chapter  Four),  FVAP’s  options,  at  least  in  the  medium  to  long  term,  appeared  to 
range  from  doing  nothing  (as  in,  leaving  the  training  to  others,  such  as  the  services)  to 
doing  it  all.  And  doing  it  could  involve  in-person,  online,  and  hybrid  delivery  modes. 

We  treated  doing  it  well  as  a  necessary  condition,2  which  was  consistent  with  both 
the  stakeholders’  view  that  FVAP  should  model  the  ideal  or  be  best  in  class  and  with 
the  call  for  greater  consideration  of  effectiveness.3  To  assess  the  agency’s  potential  for 
excellence  and  as  a  basis  for  developing  recommendations  on  program  improvements, 
we  defined  doing  it  well  in  terms  of  established  principles  of  adult  learning  and  train¬ 
ing  evaluation. 

We  summarize  the  principles  and  best  implied  practices  in  Table  E.l  and  discuss 
them  in  more  detail  in  Appendix  F. 

In  our  exploration  of  the  literature,  we  found  that  the  methods  to  address  these 
principles  are  often  mutually  reinforcing,  such  as  in  the  following: 

•  Well-crafted  periods  of  active  learning  can  serve  to  reaffirm  learning  objectives, 
motivate  learning,  tap  experience,  speak  to  diversity,  hold  attention,  and  promote 
retention. 

•  Evaluation  that  speaks  to  behavior  and  results  forces  early  consideration  of  learn¬ 
ing  objectives  in  relation  to  program  design  and  training  methods. 


2  In  effect,  we  were  arguing  that,  if  FVAP  does  not  have  (or  have  a  means  to  obtain)  the  capabilities  and  capaci¬ 
ties  to  do  something  well,  it  should  consider  walking  away. 

3  In  this  way,  we  also  departed  from  our  approach  to  the  requirements  analysis  in  phase  1,  in  which  we  consid¬ 
ered  whether  FVAP  was  covering  a  requirement  but  not  whether  it  was  covering  it  adeptly. 
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Table  E.1 

Learning  and  Evaluation  Principles 

Principle 

implied  Best  Practice3 

Adult  learners  need 

Suggesting  the  importance  of 

To  know  why  they  are  learning 

Spelling  out  purpose  and  objectives 

To  see  applicability  to  problem-solving  for 
motivation 

Stressing  applicability 

Instructional  approaches  that 

Respect  and  build  on  their  experience 

Drawing  on  trainees'  experience 

Match  their  diverse  backgrounds  and 
learning  styles 

Appealing  to  multiple  learning  styles  (e.g., 

VARK) 

Active  involvement  in  the  learning  process 

Giving  trainees  some  control  over  their  learning 
experience 

In  general,  most  or  all  learners  need 

Instructional  approaches  that  address  varied 
and  limited  attention  spans 

Keeping  it  lively  and  mixing  it  up 

Engagement  in  and  with  learning  materials 

Engaging  trainees  in  and  with  materials 

Evaluation  should  be  embedded  in  program 
training  design  from  the  outset 

Using  "four  stages"  (reaction,  learning,  behavior, 
and  results)  to  guide  evaluations 

SOURCES:  Principles  adapted  primarily  from  Bryan,  Kreuter,  and  Brownson,  2009;  Teaching  Center, 

2013;  and  D.  Kirkpatrick,  1978;  best  practices  based  on  RAND  staff  analysis  of  those  documents. 

NOTE:  VARK  =  visual,  aural,  read  and  write,  and  kinesthetic. 

a  The  set  of  principles  and  practices,  taken  in  its  entirety,  also  suggests  the  importance  of  creating 
opportunities  for  trainee  participation  and  giving  trainees  opportunities  to  participate. 

In  Appendix  F,  we  derive  and  present  a  set  of  detailed  recommendations4  for 
implementation,  tailored  to  FVAP’s  circumstances,  and  describe  our  method  of  trans¬ 
ferring  them  to  FVAP  in  a  daylong  workshop  that  we  led. 

In  that  appendix,  we  also  present  a  sample  evaluation  form,  consistent  with  the 
learner-centered  approach  to  participant  feedback  (D.  Kirkpatrick,  1978). 

In  considering  whether  FVAP  is  or  could  be  doing  it  well,  we  found,  through  our 
observations  of  training  across  services  and  venues,  that  FVAP  was  doing  some  things 
(such  as  stressing  applicability)  better  than  others  but,  with  some  investment  in  the 
program  (see  below  and  Appendix  F),  could  excel  more  uniformly.  We  suggested  pro¬ 
fessional  development  (e.g.,  in  the  form  of  a  trainer-training  program)  as  a  means  to  fill 
the  gaps  in  capability  and  capacity,  and  we  pointed  FVAP  to  a  set  of  professional  asso- 


4  These  recommendations  are  also  consistent  with  TRADOC  PAM  525-8-2.  The  pamphlet  describes  a  learner¬ 
centric  learning  model,  focused  on  context-based,  collaborative,  problem-centered  instruction. 
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ciations  and  websites  that  focus  on  such  programs  and  provide  downloadable  instruc¬ 
tional  materials. 


Weighing  the  Evidence 

We  approached  the  analysis  of  options  for  the  VAO  training  program  in  two  steps: 
First,  we  assessed  the  benefits,  costs,  and  risks  of  adopting  the  implied  best  practices; 
second,  assuming  the  adoption  of  best  practices,  we  compared  the  notional  benefits, 
costs,  and  risks  of  each  training  option,  framed  in  terms  of  in-person  and  online  train¬ 
ing  modes. 

To  conduct  the  analysis,  we  drew  on  the  project-management  tools  (Appendix  D); 
on  our  site  visits  and  direct  observations  of  training;  on  our  conversations  with  YAOs 
and  IVA  office  staff;  and  on  additional  conversations  with  FVAP  staff  during  the  day¬ 
long  workshop.  In  the  course  of  the  workshop,  we  sought  feedback  from  FYAP  staff 
and,  a  testament  to  the  value  of  the  interaction,  worked  with  their  comments  in  the 
analysis  of  options. 

Figure  E.l  describes  the  two  steps  as  elements  of  a  systematic  decisionmaking 
process.  (See  Appendix  D  for  a  full  articulation  of  the  process.) 

On  balance,  the  best  practices  looked  promising.  The  benefits  of  adopting  them 
could  be  substantial  and  recurring  (e.g.,  in  terms  of  learning  and  retention  and,  hence, 
capability  and  capacity),  and  the  costs  appeared  be  modest  and  mostly  one-time,  con¬ 
sisting  of 

•  professional  development 

•  modifications  to  course  materials 

•  redeployment  of  materials. 

Historically,  FVAP  has  maintained  a  primary  group  of  about  four  trainers  and 
sent  one  member  of  that  group  to  each  training  event;  more  recently,  FVAP  has  sent 
two-person  teams,  consisting  of  one  primary  trainer  and  one  other  staff  person.  If 
extending  professional  development  to  the  primary  group  (and  to  whoever  manages 
the  program),  the  costs  might  amount  to  four  or  five  tuitions  in  a  two-  or  three-day 
training  program  and  the  associated  staff  time.  (Members  of  the  primary  group  could 
act  as  mentors  to  other  training  staff.)  Additionally,  FVAP  might  sponsor  course  work 
in  training  evaluation  and  presentation-making.  Given  that  FVAP  retools  its  instruc¬ 
tional  materials  for  each  general-election  cycle,  it  would  be  dedicating  resources  to 
modifications  and  redeployment,  regardless.  Admittedly,  this  round  of  modifications 
might  require  more  effort  than  prior  rounds,  but,  once  implemented,  the  shift  would 
be  complete  and  future  rounds  should  require  no  more  effort  than  usual. 
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Figure  E.1 

Elements  of  a  Systematic  Decisionmaking  Process 


Don't  know 
(refine  analysis) 


RAND  RR882-E.  1 


We  saw  few  downside  risks  to  adopting  best  practices.  A  member  of  the  primary 
group  might  leave  FVAP  and  take  his  or  her  human  capital  along,  but,  if  he  or  she 
were  acting  as  a  mentor  to  other  staff  or  a  replacement,  at  least  some  capital  would  be 
transferred  before  his  or  her  departure. 

Taking  the  adoption  of  best  practices  as  given,  we  then  asked,  “How  do  the  ben¬ 
efits,  costs,  and  risks  of  in-person  and  online  training  modes  compare?” 

Tables  E.2,  E.3,  and  E.4  set  out  the  results  of  our  analysis  of  the  options,  based 
on  evidence  drawn  from  our  site  visits  and  from  our  own  encounters  with  FVAP’s 
training. 

We  found  the  following: 

•  The  benefits  of  in-person  training,  much  more  so  than  of  online  training,  can 
extend  beyond  those  of  the  initial  learning  experience  and  spread  across  the 
system  through  networking,  signaling,  and  other  methods,  but  online  training 
can  be  accessed  globally,  24/7. 

•  The  costs  of  in-person  training  are  likely  to  be  greater  than  those  of  online  train¬ 
ing,  especially  if  IT  is  already  in  place  and  available  on  installations. 

•  Each  mode  presents  risks  to  the  individual  learning  experience  insomuch  as  it 
might  fail  to  meet  the  learning  needs  of  people  who  prefer  the  other  mode,  pres¬ 
ent  challenges  of  availability,  or  entail  environmental  distractions,  but  some  insti¬ 
tutional  and  systemic  risks  pertaining  to  coordination  and  turnout  are  specific  to 
in-person  training. 
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Table  E.2 

Benefits  of  In-Person  and  Online  Training 


Benefit  Type 

In-Person  Training 

Online  Training 

Individual 

• 

Facilitates  real-time  interaction  among 

•  Can  be  self-paced 

learning 

participants,  e.g.,  question-and-answer 

•  Can  be  offered  24/7 

experience 

sessions 

•  Can  be  available  as  a  refresher 

• 

Meets  the  needs  of  those  who  prefer  in- 
person  training  and  others  less  so 

•  Meets  the  needs  of  those  who 
prefer  online  training  and 
others  less  so 

Institutional  or 

• 

Promotes  networking  with  and  among 

•  Flolds  potential  for  substantial 

systemic 

VAOs  via  direct  contact  and  interaction 

domestic  and  international 

• 

Raises  FVAP's  profile  as  a  voting  assistance 
resource 

reach 

•  Implies  exposure  to  a  web- 

• 

Signals  importance  of  assistance  and 

VAOs'  role  as  provider 

based  voting  assistance 
resource 

• 

Presents  (unbounded)  opportunity  to 
gather  information,  obtain  feedback,  and 
deliver  message  on  voting  assistance 

•  Presents  (bounded)  opportu¬ 
nity  to  gather  information, 
obtain  feedback,  and  deliver 
message  on  voting  assistance 

SOURCE:  RAND  staff  analysis. 

NOTE:  The  analysis 

assumes  adoption  of  best  practices. 

Table  E.3 

Costs  of  In-Person  and  Online  Training 

Cost  Type 

In-Person  Training 

Online  Training 

Coordination  and 

Training  materials 

•  Training  materials 

preparation 

Shipping  fees 

Interactions  with  services  and  installa¬ 
tions  and  related  goodwill 

Facility  preparation 

•  IT  acquisition,  if  not  already 
available;  development;  and 
support 

Travel 

FVAP  trainers 

In  some  instances,  IVAOs  and  trainees 

•  Not  applicable 

Time  allocation 

FVAP  and  FVAP  trainers 

SVAOs,  installations  (e.g.,  facility  main¬ 
tenance  and  audio-visual  crews),  IVAOs, 
and  trainees 

•  Trainees 

•  Staff  charged  with  supporting 

IT  (installations,  DoD,  FVAP)  and 
the  training  program  more  gen¬ 
erally  (FVAP) 

Opportunity  costs 

• 

Net  benefits  of  alternatives 

•  Net  benefits  of  alternatives 

SOURCE:  RAND  staff  analysis. 

NOTE:  The  analysis  assumes  adoption  of  best  practices. 


The  analysis  also  supported  continuance  of  a  mixed  strategy  of  in-person  and 
online  training  and  implementation  of  programmatic  improvements.5  The  combina- 


5  In  a  manner  that  is  consistent  with  our  recommendation  for  a  mixed  strategy,  Army  training  doctrine  describes 
the  need  for  blended  learning.  Blended  learning  is  the  combination  of  online  or  technology-delivered  instruc¬ 
tion  and  face-to-face  instruction:  “It  blends  the  efficiencies  and  effectiveness  of  self-paced,  technology-delivered 
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Table  E.4 

Risks  of  In-Person  and  Online  Training 


Risk  Type 


In-Person  Training 


Online  Training 


Individual  learning 
experience  (see 
below,  also) 


Institutional  or 
systemic 


Failure  to  meet  learning  needs  of  those 
who  prefer  online  training 
Unavailability  of  trainees 
Distractions  in  training  site 
Inappropriately  configured  site 


Coordination  or  preparation  failure,  poten¬ 
tially  affecting  institutional  relationships, 
accrual  of  ancillary  benefits,  and  learning 
experience 

Weak  or  wrong  turnout3  potentially  affect¬ 
ing  institutional  relationships,  accrual  of 
ancillary  benefits,  and  learning  experience 


Failure  to  meet  learning 
needs  of  those  who  prefer 
in-person  training 
Inaccessibility  of  IT 
Distractions  in  training  site 
(e.g.,  shared  equipment 
and  IT  bullpen) 

IT  failures,  potentially 
affecting  institutional  rela¬ 
tionships,  accrual  of  ancil¬ 
lary  benefits,  and  learning 
experience 


SOURCE:  RAND  staff  analysis. 

NOTE:  The  analysis  assumes  adoption  of  best  practices. 

a  In  this  context,  wrong  would  involve  the  attendance  of  a  VAO  who  is  soon  rotated  out  to  a  new 
position. 


tion  of  benefits,  costs,  and  risks  pointed  to  the  continuance  of  both  modes  of  training, 
responsive  to  differences  in  learning  styles,  differences  in  service  cultures  and  settings, 
the  potential  for  ancillary  gains,  and  the  risks  of  competing  demands  on  personnel  and 
of  staff  turnover,  especially  among  ADM  staff.  Moreover,  it  suggested  several  avenues 
of  improvement,  including  the  possibilities  of  rebalancing  through  streamlining,  tar¬ 
geting,  and  tailoring  and  working  with  local  power  brokers  to  mitigate  the  risks  of 
coordination  failures.  We  also  identified  opportunities  to  build  ancillary  gains  into  the 
training  program.  Lastly,  the  call  to  do  it  well  and  the  initial  analysis  of  the  costs,  ben¬ 
efits,  and  risks  of  adopting  best  practices  suggested  both  the  adoption  of  best  practices 
and  a  need  for  related  professional  development.  In  the  next  section,  we  discuss  these 
recommendations  in  greater  detail. 


Recommendations  and  Guidance  for  Voting  Assistance  Officer 
Training 


We  concluded  our  analysis  of  the  VAO  training  program  in  phase  2  with  a  set  of  spe¬ 
cific  suggestions  for  strengthening  the  VAO  training  program.  In  broad  terms,  we 
suggested  that  FVAP  “maintain  [its]  training  profile,  but  do  it  better  and  smarter,”  by 


instruction  with  the  expert  guidance  of  a  facilitator,  and  can  include  the  added  social  benefit  of  peer-to-peer 
interactions”  (U.S.  Department  of  the  Army,  2011,  p.  19).  See  U.S.  Department  of  the  Army,  2011,  for  more 
information. 
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•  adopting  best  practices 

•  streamlining  in-person  engagement 

•  reducing  risks  of  coordination  failures  with  training  venues 

•  building  the  attainment  of  ancillary  gains  into  the  training  program. 

Adopt  Best  Practices 

Our  recommendations  on  best  practices,  which  we  describe  in  more  detail  in  Appen¬ 
dix  F,  largely  relate  to  learning  objectives,  interactivity,  knowledge  transfer,  and  the 
reconceptualization  of  training  evaluations.  We  recognized  that  making  in-person 
training  more  interactive  (e.g.,  with  discussions,  simulations,  case  studies,  and  group 
activities)  would  require  a  paradigm  shift.  Trainers  would  need  to  know  more  about 
their  trainees;  allow  them  opportunities  (with  appropriate  tonal  cues)  to  participate; 
and  learn  to  watch,  gauge,  and  adapt  to  their  reactions.  Appropriate  tonal  cues  could 
include  repeating  back  and  making  use  of  trainees’  questions  and  comments.  For 
example,  if  the  trainer  asked  trainees  whether  they  had  encountered  a  particular  prob¬ 
lem  as  either  VAOs  or  UOCAVA  voters,  the  trainer  could  explicitly  acknowledge  the 
responses  and  then  try  to  weave  them  into  the  training  session.  To  succeed,  trainers 
would  need  to  be  able  to  train  responsively  but  stay  on  track. 

Our  suggestions  tied  back  to  the  overarching  themes  of  phase  1,  regarding  two- 
way  communication  (e.g.,  do  not  just  broadcast  to  stakeholders  but  also  receive,  pro¬ 
cess,  and  respond  to  the  information  and  feedback  they  provide).  Also  harking  back  to 
a  theme  of  phase  1,  we  recommended  that  FVAP  invest  in  professional  development 
for  staff  members  most  likely  to  direct  the  program  or  engage  in  training  repeatedly. 

Streamline  In-Person  Engagement 

We  recommended  that  FVAP  offer  regional  and  targeted  training  sessions,  develop 
audience-specific  training  modules,  and  cultivate  relationships  with  alternative  provid¬ 
ers. 

With  more  regional  and  better-targeted  training  sessions,  FVAP  could  visit  fewer 
installations  overall  and  get  more  from  them.  For  example,  it  might  identify  domestic 
and  overseas  hot  spots,6  large  markets,  and  other  high-impact  markets  and  use  train¬ 
ing  as  an  opportunity  both  to  improve  faltering  relationships  and  to  maintain  healthy 
and  productive  relationships.  In  visiting  fewer  installations,  but  with  greater  purpose, 
FVAP  might  reduce  its  operating  costs  (and  those  of  the  services)  and  still  increase  the 
benefits  of  the  program. 

We  also  suggested  developing  train-the-trainer  and  IVAO-specific  training  mod¬ 
ules  for  online  and  in-person  delivery.  In  our  discussions  with  IVAOs,  we  found  that 
some  IVAOs  were  offering  impromptu  training  sessions  to  newly  appointed  UVAOs 


6  The  dashboard  under  development  in  a  related  RAND-FVAP  project  could  provide  some  of  the  data  for  this 
analysis  and  has  been  considered  by  FVAP  for  this  purpose. 
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and  that  an  FVAP-provided  package  to  guide  them  and  ensure  consistency  would 
be  valuable.  Moreover,  we  noted  that  the  FVAP  training  sessions  both  in-person  and 
online  were  geared  to  UVAOs  and  that  IYAOs,  who  fill  a  somewhat  different  role, 
would  benefit  from  training  oriented  to  meet  their  particular  needs  as  installation-level 
coordinators,  conduits,  and  resources. 

Lastly,  we  recommended  cultivating  relationships  with  installation-based  spou¬ 
sal  and  dependent  organizations  and  other  NGOs,  including  those  overseas,  to  lessen 
the  burden  on  FVAP  trainers  and  training.  In  working  through  intermediaries,  such 
as  NGOs,  FVAP  might  forgo  some  of  the  ancillary  benefits  of  direct  engagement,  but 
it  might  also  be  able  to  extend  its  reach  to  meet  underlying  knowledge  requirements. 

Reduce  Risks  of  Coordination  Failures  with  Training  Venues 

To  address  coordination  failures,  such  as  inadequate  site  preparation  and  weak  turn¬ 
out,  we  suggested  that  FVAP  ramp  up  its  efforts  to  identify  and  work  with  power  bro¬ 
kers  on  and  off  installations.  Across  venues,  we  observed  the  positive  effects  of  issuing 
training  invitations  from  appropriate  authority  figures,  using  institutionally  appropri¬ 
ate  vocabulary.  In  these  cases,  attendance  was  high;  in  other  cases,  it  was  not.  Our 
sample  was  small,  but  the  observation  was  striking.  As  a  related  matter,  we  also  empha¬ 
sized  the  importance  of  working  within  institutional  norms  and  at  appropriate  institu¬ 
tional  levels.  In  some  instances,  it  might  be  necessary  for  the  director  to  reach  out  to 
his  or  her  counterpart  or  bureaucratic-hierarchical  equivalent;  in  others,  it  should  be 
sufficient  to  operate  at  the  staff  level. 

Build  the  Attainment  of  Ancillary  Gains  into  the  Training  Program 

In  formulating  this  recommendation,  we  took  as  a  given  that  ancillary  benefits  would 
not  accrue  automatically  and  would  require  some  additional  thought,  planning,  and 
action  on  the  part  of  FVAP.  We  addressed  four  areas — namely,  networking,  signaling, 
resource  exposure,  and  information  collection: 

•  Use  training  to  build  stronger  networks  by  encouraging  interaction  among  train¬ 
ees,  both  during  and  after  training;  encouraging  repeat  performers  at  all  levels 
(IVAOs,  VAO  office  staff,  and  IVAOs);  and  emphasizing  the  value  of  maintain¬ 
ing  and  passing  down  continuity  folders,  which,  under  current  policy,  should 
include  lessons  learned  and  are  required  of  all  VAOs.  Through  interaction,  during 
training  sessions,  and,  perhaps,  through  online  communities  of  learning,  train¬ 
ees  would  know  the  identities  of  others  facing  similar  issues  who  could,  in  turn, 
serve  as  resources  through  shared  experience.  Regarding  repeat  performers,  we 
suggested  the  potential  benefits  of  encouraging  trainees  to  take  on  similar  roles  in 
the  future,  in  other  venues.  Thus,  training  would  not  be  a  one-shot  deal  with  little 
or  no  carry  over,  and  FVAP  might  be  able  to  develop  a  cadre  of  well-informed, 
seasoned,  and  able  service  providers. 
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•  Use  physical  presence  to  signal  the  importance  of  voting  assistance  through  meet- 
and-greets  and  site  visits,  both  on  and  off  base.  Although  we  recognize  the  logisti¬ 
cal  challenges  of  coordinating  training  sessions  across  multiple  venues,  it  did  not 
appear  as  if  FVAP  was  reaching  out  systematically  to  IVAOs  and  IVA  office  staff 
or  local  election  officials  to  make  the  most  of  face-to-face  contact  with  installa¬ 
tions  and  communities. 

•  Use  training  events — in  person  and  online — to  increase  trainees’  direct  exposure 
to  FYAP  resources.  This  could  be  accomplished  through  simulated  links  in  in- 
person  training  and  through  direct  links  embedded  in  exercises  in  online  train¬ 
ing. 

•  Intentionally  gather  and  obtain  information,  including  direct  feedback  from 
intermediaries,  and  develop  mechanisms  to  capture  and  use  new  knowledge  pro¬ 
ductively.  For  example,  FVAP  could,  in  collaboration  with  the  services,  develop 
mechanisms  to  capture  lessons  from  continuity  folders  and  share  them  intergen- 
erationally,  across  installations,  and  even  across  services.  It  could  also  maintain 
a  shared  database  of  insights,  comments,  and  other  notes  drawn  from  the  field. 

In  addition,  we  noted  that,  if  FVAP  were  to  streamline  its  engagement,  it  might 
need  fewer  resources  to  use  the  training  sessions  more  productively.  For  example, 
FVAP  would  need  to  arrange  meet-and-greets  at  fewer  installations — in  general,  the 
streamlining  would  reduce  the  requirement  for  advance  preparation  and  pretraining 
reconnaissance-like  activities.  At  the  same  time,  streamlining  might  imply  less  work 
for  the  services. 

Lastly,  and  speaking  to  the  broader  themes  of  phase  1,  we  suggested  that  FVAP 
view  training  as  an  opportunity  to  spread  its  message  consistently  across  venues.  To  do 
so,  it  would  need  to  reevaluate  its  in-person  and  online  training  materials  as  outreach 
and  take  steps  to  ensure  that  the  training  materials  convey  the  same  message  as  other 
outreach  materials. 


APPENDIX  F 


Adult  Learning  Principles  and  Training  Evaluation 


The  law  affords  FVAP  considerable  flexibility  regarding  training  modes  and  methods. 
Currently,  FVAP  offers  both  online  and  in-person  training  for  VAOs.  To  inform  rec¬ 
ommendations  on  the  modality  of  FVAP’s  training  program,  we  began  by  reviewing 
literature  on  the  effectiveness  of  online  training,  in  both  absolute  terms  and  relative 
to  the  effectiveness  of  in-person  training.  The  literature  speaks  to  potential  benefits 
(largely  relating  to  costs,  convenience,  and  reach)  and  to  efficacy,  in  terms  of  knowl¬ 
edge  acquisition. 


Benefits  and  Efficacy  of  Online  Training 

Numerous  studies  (e.g.,  Gu  et  ah,  2012;  Means  et  ah,  2010;  Strother,  2002)  cite  advan¬ 
tages,  including  economic  benefits,  convenience,  flexibility,  self-paced  learning,  stan¬ 
dardized  delivery,  and  the  potential  for  greater  reach,  and  some  suggest  limitations.1 
Among  the  economic  benefits  are  that  online  training  can  reduce  travel  expenses  for 
trainers  and  trainees,  reduce  the  amount  of  time  that  trainees  must  be  off  the  job,  and 
reduce  the  shipping  fees  associated  with  sending  training  materials  to  training  events. 
Trainees  can  take  online  courses  at  their  convenience  and  learn  at  their  own  pace  rather 
than  wait  for  a  scheduled  in-person  training  session.  Online  courses  are  also  not  sub¬ 
ject  to  differences  in  trainers’  teaching  styles.  However,  Straus  et  al.,  2014,  reports  less 
satisfaction  with  opportunities  for  performance  feedback  and  a  desire  for  more  peer 
interaction. 

The  evidence  as  to  the  efficacy  of  online  training,  specifically  in  comparison  to  in- 
person  instruction,  is  suggestive.  Means  et  al.  conducted  a  systematic  review  of  empiri¬ 
cal  studies  of  online  learning  from  1996  through  July  2008  for  the  U.S.  Department 
of  Education  and  found  that,  with  the  possible  exception  of  hybrid  classes  (although 
their  sample  included  only  three  hybrid  classes),  there  are  generally  no  significant  dif¬ 
ferences  in  student  attainment  of  learning  outcomes  associated  with  the  different  deliv- 


1  We  address  the  potential  benefits  and  costs  of  online  and  in-person  training  in  more  detail,  drawing  evidence 
from  field  observations,  in  Chapter  Six. 
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ery  methods  (i.e.,  online  learning  and  face-to-face  instruction),  even  when  controlling 
for  course  and  instructor  (Means  et  ah,  2010).  Their  findings  are  consistent  with  the 
general  conclusions  of  prior  research  that  indicate  that  student  performance  in  online 
and  traditional  classroom  versions  of  the  same  courses  is  similar  (Means  et  ah,  2010). 
According  to  Merrill  and  Galbraith,  2010,  p.  20,  “Prior  empirical  research  appears  to 
indicate  that,  in  general,  student  performance  in  on-line  and  traditional  classroom 
versions  of  the  same  course  is  similar.  This  conclusion  appears  consistent  across  many 
professional  disciplines.  .  .  .”2 

Some  students  might  prefer  one  method  over  the  other,  given  differences  in  their 
learning  styles  (see  below),  but,  on  balance,  one  might  infer  that  the  methods  are 
roughly  equivalent  in  efficacy  in  terms  of  learning  for  an  average  mix  of  students. 

Moreover,  in-person  training  might  also  offer  ancillary  benefits  that  were  not  a 
focus  of  the  literature  (e.g.,  with  respect  to  networking  and  signaling).3 

Taken  together,  the  flexibility  of  the  law  regarding  training  modes  and  meth¬ 
ods  and  the  plausible  claim  of  no  significant  difference  in  efficacy  present  FYAP  with 
options.  The  options  for  its  training  program  range  from  doing  nothing  to  doing  it 
all,  with  doing  it  being  training  in  person,  online,  or  a  combination  of  the  two  (i.e.,  a 
hybrid  approach).  This  decision  depends  on  several  factors,  including  the  balance  of 
benefits,  costs,  and  risks  to  FVAP  and  others,  along  with  whether  FVAP  can  do  it  well. 

But  what  does  it  mean  to  do  it  well? 


"Doing  It  Well" 

To  determine  what  it  means  to  do  it  well,  we  examined  principles  of  adult  learning  that 
focus  on  how  adults  learn  and  how,  as  a  consequence,  to  train  them  effectively.  These 
principles  rest  on  a  variety  of  theories,  models,  and  explanations  of  how  adults  learn, 
which,  although  differing  in  emphasis,  yield  a  general  consensus  on  central  tenets 
(see  Bryan,  Kreuter,  and  Brownson,  2009,  p.  558,  for  an  overview).  The  field  of  adult 
learning  originated  in  the  mid-20th  century,  when  researchers  and  adult  educators 
began  to  consider  the  differences  in  learning  in  adulthood  and  in  childhood.  In  1968, 
Malcolm  Knowles  proposed  the  adoption  of  the  theory  of  andragogy  (Merriam,  2001). 
Andragogy,  which  is  “the  art  and  science  of  helping  adults  learn,”  is  premised  on  four 
assumptions  (Knowles,  1980,  pp.  44-45): 

These  assumptions  are  that  as  individuals  mature:  1)  their  self-concept  moves  from 
one  of  being  a  dependent  personality  toward  being  a  self-directed  human  being; 

2)  they  accumulate  a  growing  reservoir  of  experience  that  becomes  an  increasingly 

2  Straus  et  al.’s  findings  on  student  self-assessments  and  grades  are  also  suggestive  but  less  conclusive,  owing  to 
fundamental  differences  in  and  confounds  among  in-person  and  online  venues  (Straus  et  al.,  2014). 

3  These  potential  benefits  are  a  focus  of  discussion  in  Chapter  Six. 
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rich  resource  for  learning;  3)  their  readiness  to  learn  becomes  oriented  increas¬ 
ingly  to  the  developmental  tasks  of  their  social  roles;  and  4)  their  time  perspective 
changes  from  one  of  postponed  application  of  knowledge  to  immediacy  of  applica¬ 
tion,  and  accordingly,  their  orientation  toward  learning  shifts  from  one  of  subject- 
centeredness  to  one  of  performance-centeredness. 

With  these  assumptions,  Knowles  suggested  several  implications  for  educational 
practice.  For  example,  building  on  the  concept  that  adults  are  self-directed,  Knowles 
emphasized  the  need  for  adult  learners  to  be  actively  involved  in  the  learning  process. 
He  suggested  engaging  adult  learners  in  a  process  of  self-diagnosis,  in  which  they  can 
assess  their  present  levels  of  competencies,  measure  the  gaps  between  their  present 
competencies  and  those  required  by  the  job,  and  identify  specific  directions  of  desir¬ 
able  growth  (Knowles,  1980).  To  build  on  adults’  experiences,  Knowles  recommended 
illustrating  new  concepts  or  broad  generalizations  with  life  experiences  drawn  from  the 
trainees  (Knowles,  1980). 

Bryan,  Kreuter,  and  Brownson,  2009,  reviewed  existing  theories  and  models  of 
andragogy  and  self-directed  learning  (Bryan,  Kreuter,  and  Brownson,  2009,  citing 
Knowles,  1975,  and  Tough,  1967,  1971)  and  synthesized  recurring  themes  into  the  fol¬ 
lowing  list  of  principles  of  adult  learning,  drawing  on  these  sources  and  many,  many 
others:4 

•  Adults  need  to  know  why  they  are  learning. 

•  Adults  are  motivated  to  learn  by  the  need  to  solve  problems. 

•  Adults’  previous  experience  must  be  respected  and  built  on. 

•  Adults  need  learning  approaches  that  match  their  backgrounds  and  identities. 

•  Adults  need  to  be  actively  involved  in  the  learning  process. 

Using  these  principles  as  a  framework,  we  then  further  synthesized  the  literature 
on  adult  learning  into  general  recommendations  for  implementation.  The  first  two 
columns  of  Table  F.l  show  the  adult  learning  principles,  with  a  few  modest  adjust¬ 
ments  (e.g.,  to  account  for  other  perspectives  in  the  literature)  and  general  recommen¬ 
dations  for  implementation.  Many  of  our  general  recommendations  are  drawn  from 
Bryan,  Kreuter,  and  Brownson,  2009,  and  Knowles,  1980,  as  well  as  from  Bjornberg, 
DellCioppia,  and  Tanzer,  2002,  and  Gu  et  al.,  2012. 

Our  recommendations  are  also  consistent  with  TRADOC  PAM  525-8-2.  The 
pamphlet  describes  a  learner-centric  learning  model,  focused  on  context-based,  collab¬ 
orative,  problem- centered  instruction: 

Classroom  learning  will  shift  from  instructor-centered,  lecture-based  methods  to  a 
learner-centered,  experiential  methodology.  Engaging  the  learners  in  collaborative 


These  principles  are  invariant  to  the  mode  (online  or  in  person)  of  delivery. 


Table  F.1 

Adult  Learning  Principles  and  Recommendations  for  Implementation 


Adult  Learning 

Principle3  General  Recommendation  for  Implementation  Illustration13 


Adults  need  to 
know  why  they  are 
learning. c 


Clearly  state  the  learning  objectives01  for  the  training  session 
as  distinct  from  topical  coverage  or  table  of  contents. 

Elicit  trainees'  reasons  for  attending  the  training  session 
(Knowles,  1980). 

Identify  necessary  competencies  (knowledge,  skills,  and 
behaviors)  to  achieve  a  given  level  of  performance  on  the  job 
(Knowles,  1980;  Bjornberg,  DellCioppia,  and  Tanzer,  2002). 
Explicitly  relate  objectives  to  trainees'  reasons  for  attending 
and  necessary  competencies. 


Use  learning  objectives  to  launch  and  anchor  the  training  ses¬ 
sion,  from  start  to  finish: 

°  State  the  learning  objectives  at  the  beginning  of  the  train¬ 
ing  session,  in  relation  to  the  necessary  competencies. 

-  Explicitly  connect  instructional  material  to  the  objectives 
throughout  the  session. 

°  Recap  the  objectives,  in  totality,  at  the  end  of  the  session. 
Begin  with  a  short  conversation  about  objectives  and  compe¬ 
tencies  (e.g.,  ask  trainees  to  identify  what  they  would  like  to 
get  from  the  training  and  then  relate  their  objectives  to  the 
stated  learning  objectives  and  necessary  competencies). 
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Table  F.1 — Continued 


Adult  Learning 
Principle3 


Adults  are 
motivated  to  learn 
by  the  need  to 
solve  problems. 


General  Recommendation  for  Implementation 


Provide  examples  of  real-life  problems  or  challenges  that 
trainees  are  likely  to  encounter  on  the  job,  and  demon¬ 
strate  how  training  will  help  them  address  those  problems  or 
challenges. 

Build  learning  experience  on  those  real-life  problems  and 
challenges  so  that  trainees  are  more  likely  to  be  engaged  in 
training,  participate  in  discussions,  and  share  insights. 


Illustration13 


•  Ask  VAOs  what  problems  or  challenges  they  have  encoun¬ 
tered  in  providing  assistance  to  UOCAVA  voters  and  what 
problems  or  challenges  they  or  their  dependents  have 
encountered  as  UOCAVA  voters. e 

•  Generate  a  discussion  around  those  issues  (e.g.,  by  asking  the 
VAOs  how  they  addressed  the  issues  and  by  considering  those 
and  other  concrete  ways  of  troubleshooting). 

•  Conduct  skill-practicing  exercises  by  giving  trainees  tricky 
scenarios  that  require  them  to  find  information  in  the  Voting 
Assistance  Guide  and  answer  questions  about  addresses  and 
other  information.^ 

•  Create  plausible  scenarios  to  introduce  and  illustrate  key 
issues,  e.g., 

-  Track  a  person  who  marries,  has  children,  buys  and  sells  a 
residence,  and  serves  overseas,  from  the  point  of  his  or  her 
enlistment  to  eventual  retirement. 

°  Develop  "what  if s"  around  political  activities  to  (1)  clarify 
the  meaning  of  the  terms  partisan  and  nonpartisan  and 
(2)  address  rules  for  particular  behaviors  among  VAOs  and 
ADM  voters  more  generally. 

•  Provide  specific  examples  of  issues  that  VAOs  typically  encoun¬ 
ter,  e.g., 

°  Determining  UOCAVA  eligibility 

-  Determining  voting  residence  address 

°  Reporting  metrics,  including  "what  counts  as  providing 
assistance"? 
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Table  F.1 — Continued 


Adult  Learning 

Principle3  General  Recommendation  for  Implementation  Illustration13 


Adults'  previous 
experience  must 
be  respected  and 
built  on. 


Relate  new  material  to  trainees'  experiences.  • 

“  Take  an  inventory  of  trainees'  experiences  as  they  relate  to 
the  learning  objectives  and  topical  coverage. 

°  Illustrate  new  concepts  or  broad  generalizations  with 
trainees'  life  experiences  (Knowles,  1980). 

Emphasize  techniques  that  can  tap  into  trainees'  • 

experiences — and,  at  the  same  time,  address  problems  or 
challenges — such  as  group  discussions,  case  studies,  simula¬ 
tion,  role-playing,  skill-practicing  exercises,  and  demonstra-  • 
tions  (Knowles,  1980). 


Ask  trainees  how  long  they  have  been  VAOs  and,  as  above, 
what  issues  they  have  encountered  in  providing  assistance  to 
UOCAVA  voters  or,  for  those  who  are  new  to  the  VAO  posi¬ 
tion,  as  UOCAVA  voters  or  as  spouses  or  parents  of  UOCAVA 
voters. 

Use  the  issues  that  VAOs  raise  as  prompts  for  a  group  discus¬ 
sion  (e.g.,  by  asking  whether  others  have  encountered  similar 
problems  and  how  they  have  handled  them). 

Have  trainees  fill  out  the  FPCA  or  FWAB  form  (using  their 
own  information)  before  reviewing  procedures  to  encourage 
discussion  of  relevant  life  experiences  (e.g.,  "has  anything 
changed  since  the  last  time  you  filled  in  this  form"  or  "has  the 
form  gotten  easier  to  fill  out  with  time  and  experience")  and 
to  identify  and  discuss  stumbling  blocks. 
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Table  F.1 — Continued 


Adult  Learning 

Principle3  General  Recommendation  for  Implementation  Illustration13 


Adults  need 
instructional 
approaches  that 
match  their  diverse 
backgrounds  and 
learning  styles. 


Present  information  and  place  it  in  context,  using  multiple 
approaches  to  appeal  to  diverse  learning  styles.9 
"  Learning-style  research  suggests  that  students  learn  dif¬ 
ferently  and  experience  higher  levels  of  satisfaction  and 
better  learning  outcomes  when  there  is  a  fit  between 
their  learning  styles  and  the  teaching  style  (Gu  et  al.,  2012, 
citing  Zuckweiler  and  Cao,  2009)  and  when  an  educa¬ 
tional  tool  is  developed  with  learning  styles  in  mind  (Gu 
et  al.,  2012,  citing  Canfield  and  Lafferty,  1974;  Kolb,  1976; 
Fleming  and  Mills,  1992;  Grasha  and  Yangarber-Hicks, 
2000). 

°  One  model  of  learning,  VARK,  incorporates  four  learning 
modes  (visual,  aural,  reading  and  writing,  and  kinesthetic) 
(Fleming  and  Mills,  1992). 

Align  online  training  with  various  learning  styles  to  increase 
perceived  ease  of  use  and  usefulness  (Gu  et  al.,  2012). 


Discuss  trainees'  learning  styles  at  the  outset  of  a  training  ses¬ 
sion  by  asking  trainees  questions  about  their  preferences  and 
relating  their  answers  to  the  types  of  approaches  that  the  ses¬ 
sion  will  include  (e.g.,  if  someone  says  he  or  she  likes  "hands- 
on"  learning,"  let  that  person  know  that  he  or  she  will  have 
an  opportunity  to  work  with  forms). 

Present  information  using  a  combination  of  approaches, 
including 

°  Visual,  involving  graphical  or  symbolic  representations 
(e.g.,  diagrams,  charts,  maps,  and  flow  charts) 

-  Aural,  involving  heard  or  spoken  information  (e.g.,  lec¬ 
tures  and  discussions) 

°  Reading  and  writing,  involving  text-based  media  (e.g., 
briefing  slides,  articles,  and  discussion  boards) 

-  Kinesthetic,  learning  by  doing  (e.g.,  simulations  and  role- 
playing  exercises). 

Try  using  simulation,  role-playing,  and  other  skill-practicing 
exercises  with  forms  (FPCA  and  FWAB)  and  various  envelopes 
or  labels  to  emphasize  reading,  writing,  and  kinesthetic  learn¬ 
ing  at  the  same  time  and  work  on  problem-solving. 

Consider  developing  modular  approaches  that  speak  to  dif¬ 
ferent  styles  and  can  be  applied,  depending  on  preference  of 
trainees  in  a  particular  session  (Bryan,  Kreuter,  and  Brownson, 
2009). 
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Table  F.1 — Continued 


Adult  Learning 

Principle3  General  Recommendation  for  Implementation  Illustration13 


Adults  need  to  be 
actively  involved 
in  the  learning 
process. 


Provide  trainees  with  • 

°  Choices  over  what  they  learn  and  how  they  learn  it,  includ¬ 
ing  presentational  methods 

°  Diagnostic  experiences  in  which  they  can  assess  their  pres¬ 
ent  levels  of  competencies;  measure  the  gaps  between 
their  present  competencies  and  those  required  by  the 
job;  and  identify  specific  directions  of  desirable  growth  • 
(Knowles,  1980) 

°  Opportunities  to  discuss  whether  training  has  met  learning  • 
objectives  (Knowles,  1980). 

Emphasize  transfer  of  learning  and  ownership  of  knowledge 
and  skills.  • 

°  Have  trainees  complete  action  plans  before  leaving  a  train¬ 
ing  session  (Bjornberg,  DellCioppia,  and  Tanzer,  2002). h 
°  Offer  refresher  sessions  as  an  opportunity  for  trainees  to 
reconvene  and  discuss  what  is  going  well  and  where  they 
are  having  problems;  sessions  can  also  provide  a  review  of 
course  content  (Bjornberg,  DellCioppia,  and  Tanzer,  2002). 

°  Encourage  trainees  to  form  networks  in  which  they  dis¬ 
cuss  strategies  for  implementing  the  training  (Bjornberg, 
DellCioppia,  and  Tanzer,  2002)  and  any  problems  or  chal¬ 
lenges  that  they  face  during  implementation. 

°  Provide  job  aids  that  serve  as  a  quick  reference  for  how  to 
use  the  knowledge  and  skills  obtained  in  the  training  ses¬ 
sion  (Bjornberg,  DellCioppia,  and  Tanzer,  2002). 


Provide  trainees  with  opportunities  to  learn  more  about 
topics 

-  In  in-person  training,  with  options  to  reexamine  difficult 
or  complex  material  and  with  leave-behinds1 
°  In  online  training,  by  clicking  on  links  to  additional 
information.-1 

Conduct  a  pretest  prior  to  the  training  session,  either  in  writ¬ 
ing  or  by  posing  a  series  of  questions  to  the  group. 

Pose  questions  as  minute  tests  at  major  transition  points  in 
the  training  session  to  determine  whether  trainees  have  taken 
in  or  processed  new  information. k 

Recap  the  learning  objectives  at  the  end  of  the  session,  and 
elicit  input  from  trainees  (through  discussion  or  feedback 
form)  about  whether  they  have  been  met. 
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Table  F.1 — Continued 


Adult  Learning 

Principle3  General  Recommendation  for  Implementation  Illustration13 

3  Adapted  from  Bryan,  Kreuter,  and  Brownson,  2009. 

b  Adapted  from  Bryan,  Kreuter,  and  Brownson,  2009,  unless  otherwise  stated. 

c  "Adults  will  spend  more  time  and  energy  learning  when  they  see  a  reason"  (Bryan,  Kreuter,  and  Brownson,  2009,  p.  559,  citing  Knowles,  Holton,  and 
Swanson,  1998,  and  Tough,  1967). 

^  Some  learners  are  goal  oriented  (Bryan,  Kreuter,  and  Brownson,  2009,  citing  Houle,  1961). 

e  It  might  be  interesting  to  probe  whether  these  were  one-time  problems  (e.g.,  associated  with  start-up  or  unusual  circumstance)  or  repeated 
problems. 

f  Depending  on  the  number  of  trainees  and  the  configuration  of  the  room,  this  can  be  done  as  a  paired  or  small-  to  medium-group  exercise  to 
encourage  interaction. 

9  Bryan,  Kreuter,  and  Brownson,  2009,  p.  561:  "[M]ost  important,  know  when  each  variant  is  appropriate  for  a  given  group  of  learners,  or  even  a 
particular  subgroup  or  individual  within  a  group." 

h  Action  plans  focus  trainees'  attention  on  the  trainees'  application  of  information  and  skills  (Bjornberg,  DellCioppia,  and  Tanzer,  2002). 

1  This  can  be  done  by  presenting  trainees  with  the  option  to  take  a  second  look  at  one  of  a  handful  of  objectives  or  subject  areas  and  polling  their 
preferences. 

J  This  feature  has  been  built  into  some  of  the  online  training  packages. 
k  This  feature  has  been  built  into  some  of  the  online  training  packages. 
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practical  and  problem  solving  exercises  that  are  relevant  to  their  work  environment 
provides  an  opportunity  to  develop  critical  21st  century  Soldier  competencies.  .  .  . 

Students  master  knowledge  and  comprehension  level  learning  objectives  outside 
the  classroom  through  individual  learning  activities  such  as  reading,  self-paced 
technology-driven  instruction,  or  research.  Collaborative  learning  activities,  dis¬ 
cussion,  identification  of  problems,  and  solving  these  problems  is  done  in  the 
small  group  classroom  environment.  This  learner-centered  instructional  approach 
encourages  student  participation  and  puts  the  instructor  in  the  role  of  a  facilitator. 
Facilitators  are  responsible  for  enabling  group  discovery.  Students  and  facilitators 
construct  knowledge  by  sharing  prior  knowledge  and  experiences,  and  by  examin¬ 
ing  what  works  and  what  does  not  work. 

Drawing  further  from  Bryan,  Kreuter,  and  Brownson,  2009,  and  others,  we  also 
derived  a  set  of  implied  best  practices  from  the  adult  learning  principles: 

•  Adults  need  to  know  why  they  are  learning:  Be  explicit  about  learning  objectives 
and  the  connection  between  learning  objectives  and  trainees’  reasons  for  atten¬ 
dance  (see  Bryan,  Kreuter,  and  Brownson,  2009,  citing  Lieb,  1991).  Spell  out  the 
purpose  and  reinforce. 

•  Adults  are  motivated  to  learn  by  the  need  to  solve  problems:  Build  learning  expe¬ 
riences  around  real-world  problems  that  learners  might  encounter  so  that  they 
are  more  likely  to  be  engaged,  participate  in  discussions,  and  share  insights  (see 
Bryan,  Kreuter,  and  Brownson,  2009,  citing  Lawler,  2003,  and  Knowles,  Holton, 
and  Swanson,  1998).  Stress  applicability. 

•  Adults’  previous  experience  must  be  respected  and  built  on:  Relating  new  mate¬ 
rial  to  existing  knowledge  and  experience  can  aid  the  learning  process  (see  Bryan, 
Kreuter,  and  Brownson,  2009,  citing  Merriam  and  Caffarella,  1999).  Draw  on 
trainees’  experience. 

•  Adults  need  instructional  approaches  that  match  their  diverse  backgrounds  and 
learning  styles:  Use  multiple  methods  of  presenting  information,  and  contextu¬ 
alize  it  in  different  ways  (see  Bryan,  Kreuter,  and  Brownson,  2009).5  Appeal  to 
multiple  learning  styles. 

•  Adults  need  to  be  actively  involved  in  the  learning  process:  Adults  are  often  self¬ 
directing  and  probably  want  some  control  over  what  they  are  learning  (see  Bryan, 
Kreuter,  and  Brownson,  2009,  citing  Knowles,  1984).  Give  trainees  some  con¬ 
trol  over  their  learning  experiences. 


’  Some  people  are  more  visual,  some  learn  better  by  reading  and  writing,  and  so  on. 
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On  the  basis  of  the  adult  learning  principles,  general  recommendations,  and 
implied  best  practices,  we  offered  FVAP  a  set  of  VAO  training-tailored  recommenda¬ 
tions,  which  we  show  in  the  third  column  of  Table  F.  1 . 

In  a  manner  that  was  consistent  with  our  overall  approach  to  this  project,  as 
one  of  ongoing,  collaborative  engagement,  we  delivered  (or  transferred)  the  principles, 
practices,  and  recommendations  in  Table  F.l  to  FVAP  in  a  daylong  event,  consisting 
of  a  general  presentation,  a  small-group  discussion,  and  a  series  of  one-on-one  meet¬ 
ings  with  FVAP  trainers.  To  augment  the  small-group  discussion,  we  also  prepared 
commentary,  questions,  and  exercises,  to  which  we  referred  as  Food  for  Thought.  The 
intent  was  to  make  the  concepts  less  abstract  and  to  help  trainers  see  how  the  concepts 
might  relate  to  their  circumstances. 

For  example,  under  the  first  principle,  “adults  need  to  know  why  they  are  learn¬ 
ing,”  we  included  the  following  questions: 

•  Can  you  identify  an  overarching  learning  objective  or  goal  for  the  training  ses¬ 
sion? 

•  What  is  the  difference  between  a  statement  of  learning  objectives  and  a  statement 
of  topical  coverage  or  a  table  of  contents? 

And  we  asked  FVAP  participants  to  compare  and  categorize  the  following  state¬ 
ments,  as  objectives  or  topics:6 

•  Today,  we  will  talk  about  continuity  folders. 

•  By  the  end  of  this  session,  you  will  understand  the  purpose  of  the  continuity 
folder. 

•  By  the  end  of  this  session,  you  will  know  how  to  construct  a  continuity  folder. 

•  This  presentation  will  provide  an  overview  of  the  forms  that  UOCAVA  voters  use. 

•  Today,  you  will  learn  how  to  address  five  common  UOCAVA  voting  mistakes. 

•  Today,  you  will  learn  the  purpose  of  each  block  of  the  FPCA  and  FWAB. 

In  addition,  we  drew  from  the  more-general  literature  on  effective  teaching  (e.g., 
Indiana  University,  2012;  Single,  1991;  Teaching  Center,  2013)  to  incorporate  two 
additional  learning  principles;  that  is,  most,  if  not  all,  learners  need  instructional 
approaches  that  address  varied  and  limited  attention  spans  and  engagement  with  mate¬ 
rial  for  retention. 

In  our  conversations  with  FVAP  and  in  our  presentation  of  the  recommenda¬ 
tions  and  illustrations  in  Box  D.l  in  Appendix  D,  we  emphasized  that  the  methods  to 
address  the  adult  and  general  learning  principles  are  often  mutually  reinforcing.  For 
example,  well-crafted  periods  of  active  learning  can  serve  to  reaffirm  objectives,  moti¬ 
vate  learning,  tap  experience,  speak  to  diversity,  hold  attention,  and  promote  retention. 


6 


We  have  edited  the  statements  slightly,  for  brevity. 
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Training  Evaluation 

Lastly,  we  looked  at  the  literature  on  training  evaluation. 

A  well-known  and  widely  used  methodology  for  evaluating  training  and  learn¬ 
ing  processes  is  the  Kirkpatrick  model.  This  model  describes  four  stages  or  steps  to 
consider  in  evaluating  a  training  program:  reaction,  learning,  behavior,  and  results, 
or  levels  1,  2,  3,  and  4,  respectively  (also  denoted  LI,  L2,  L3,  and  L4,  respectively) 
(D.  Kirkpatrick,  1978). 

Table  F.2  describes  each  stage  and  measurement  methods.  By  contrast,  current 
FVAP  training  evaluations  have  focused  mostly  on  LI,  with  some  consideration  of  L2. 
In  our  conversations  with  FVAP,  we  noted  that  reaching  out  to  L3  and  L4  can  encour¬ 
age  early  consideration  of  learning  objectives  in  relation  to  program  design  and  train¬ 
ing  methods. 

LI  evaluation  forms,  including  those  used  previously  by  FVAP,  are  often  trainer- 
centered  or  program-centered  rather  than  learner-centered.  For  example,  a  participant 
might  be  asked  to  rate  such  statements  as  these: 


•  The  program  objectives  were  clearly  defined. 

•  The  program  objectives  were  covered  by  the  instructor. 

•  The  material  was  the  right  level  of  complexity  for  my  background. 

Following  in  his  father’s  footsteps,  Jim  Kirkpatrick  introduced  the  concept  of 
a  new  learner-centered  LI  evaluation  form  (J.  Kirkpatrick,  undated).  Using  this  as  a 


Table  F.2 

Kirkpatrick  Levels  of  Evaluation 


Level 

Criterion 

Basis 

Measurement  Method 

LI 

Reaction 

Extent  to  which  participants  react 
favorably  to  training 

•  How  do  participants  feel  about  the 
program? 

•  How  satisfied  are  they  with  the 
training? 

Participant  feedback  or  reaction  form 
(learner  centered) 

L2 

Learning 

Extent  to  which  participants  acquire 
intended  knowledge,  skills,  attitudes, 
confidence,  and  commitment  from 
training 

Measure  knowledge,  skills,  and  attitudes  in 
pretest  and  posttest 

L3 

Behavior 

Extent  to  which  participants  apply  what 
they  learned  during  training  when  they 
are  on  the  job  and  change  their  behavior 
on  the  job 

Measure  changes  in  on-the-job  behavior 
(e.g.,  with  interviews) 

L4 

Results 

Extent  to  which  targeted  outcomes 
result  from  the  training  program  and 
subsequent  reinforcement 

Measure  conditions  before  the  training 
program  and  compare  with  conditions 
after  the  program 

SOURCE:  D.  Kirkpatrick,  1978. 
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model,  we  drafted  a  new  evaluation  form  for  FVAP  to  use  to  elicit  participant  feedback 
for  its  in-person  training  sessions.  This  evaluation  form  is  shown  in  Figures  F.l  and  F.2. 


184  The  Federal  Voting  Assistance  Program  and  the  Road  Ahead 


Figure  F.1 

Proposed  Training  Evaluation  Form,  Page  1 


DRAFT  VAO  Training  Evaluation  Form  June  25,  2014 


Strongly 

Disagree 

(1) 

Disagree 

(2) 

Neutral 

(3) 

Agree 

(4) 

Strongly 

Agree 

(5) 

1.  Learning  objectives 

(a)  1  understood  the  learning  objectives 

(b)  1  obtained  knowledge  and  skills  that  are  consistent  with 
the  learning  objectives 

(c)  1  am  clear  about  what  is  expected  of  me  as  a  VAO  as 
result  of  taking  this  training  course 

II.  Course  materials 

(a)  1  found  the  course  materials  (e.g.,  slides,  handouts,  and 
other  VAO  resources)  easy  to  follow  or  navigate 

(b)  1  found  the  complexity  and  level  of  detail  of  the  material 
appropriate  to  my  background  and  experience 

(c)  1  believe  that  the  course  materials,  including  resources, 
will  be  essential  to  my  success  as  a  VAO 

III.  Content  relevance 

(a)  1  believe  that  1  will  be  able  to  apply  what  1  learned  today 
in  my  role  as  a  VAO 

(b)  1  believe  that  1  have  obtained  the  necessary  knowledge 
and  skills  to  be  a  successful  VAO 

(c)  1  am  dear  about  where  to  find  answers  to  the  questions 
that  will  arise  in  my  role  as  a  VAO 

IV.  Facilitator  knowledge 

(a)  My  learning  was  enriched  by  the  facilitator's  knowledge 

(b)  My  learning  was  enriched  by  the  experiences  and 
examples  that  the  facilitator  shared 

V.  Facilitator  delivery  and  style 

(a)  1  was  well  engaged  during  the  training  session 

(b)  1  found  it  easy  to  be  actively  involved  during  the  session 

(c)  1  had  ample  opportunity  to  ask  questions  and  receive 
answers  to  my  questions  during  the  session 

(d)  1  had  ample  opportunity  to  practice  or  demonstrate  the 
skills  that  1  was  asked  to  learn  during  the  session 

(e)  1  was  comfortable  with  the  pace  of  the  session 

(f)  1  was  comfortable  with  the  length  of  the  session 

VI.  Facility  and  environment 

1  found  the  room  and  set  up  to  be  comfortable,  free  of 
distractions,  and  conducive  to  learning 

Using  the  space  on  the  back  of  this  form,  please: 

(1)  Explain  any  items  rated  as  "Disagree”  or  "Strongly  Disagree." 

(2)  List  the  three  most  important  things  you  learned  from  this  training  course. 

(3)  Tell  us  how  we  can  strengthen  or  improve  the  training  course. 


SOURCE:  Developed  by  RAND  staff,  based  on  J.  Kirkpatrick,  undated. 

RAND  RR882-F.  1 
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Figure  F.2 

Proposed  Training  Evaluation  Form,  Page  2 


DRAFT  VAO  Training  Evaluation  Form  June  25,  2014 


(1)  Explain  any  items  rated  as  "Disagree"  or  "Strongly  Disagree." 
(Please  include  the  item  number,  e.g.,  "I.(a)_") 


(2)  List  the  three  most  important  things  you  learned  from  this  training  course. 


(3)  Tell  us  how  we  can  strengthen  or  improve  the  training  course. 


SOURCE:  Developed  by  RAND  staff,  based  on  J.  Kirkpatrick,  undated. 

RAND  RR882-F.2 
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